2023-11-28T11:38:54.599 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T11:38:54.600 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_neesingh@teuthology --verbose --archive /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964 --name neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi -- /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/orig.config.yaml 2023-11-28T11:38:54.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:38:54.733 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469964' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: 17c45c858c4cb3105f973d12c6c74f8d5037aa89 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4140 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 17c45c858c4cb3105f973d12c6c74f8d5037aa89 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_11:27:19 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T11:38:54.733 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa; will attempt to use it 2023-11-28T11:38:54.734 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks 2023-11-28T11:38:54.734 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T11:38:54.736 INFO:teuthology.task.internal:Checking packages... 2023-11-28T11:38:54.761 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T11:38:54.761 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T11:38:54.761 INFO:teuthology.packaging:ref: None 2023-11-28T11:38:54.761 INFO:teuthology.packaging:tag: None 2023-11-28T11:38:54.761 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T11:38:54.761 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:38:54.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T11:38:54.979 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T11:38:54.979 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T11:38:54.986 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T11:38:54.986 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T11:38:55.008 INFO:teuthology.task.internal:Saving configuration 2023-11-28T11:38:55.024 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T11:38:55.032 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:38:55.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:33:48.971435', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:38:55.083 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:33:48.969704', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:38:55.083 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T11:38:55.090 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T11:38:55.090 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T11:38:55.090 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T11:38:55.172 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f572684ea60>, signals=[15]) 2023-11-28T11:38:55.172 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T11:38:55.181 INFO:teuthology.task.internal:Opening connections... 2023-11-28T11:38:55.181 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:38:55.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:38:55.253 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:38:55.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:38:55.325 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T11:38:55.332 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2023-11-28T11:38:55.352 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-11-28T11:38:55.353 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:ID="rhel" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="fedora" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8.6" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T11:38:55.409 INFO:teuthology.orchestra.run.smithi144.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T11:38:55.410 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T11:38:55.410 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:38:55.410 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T11:38:55.410 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T11:38:55.410 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T11:38:55.410 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T11:38:55.411 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-11-28T11:38:55.431 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2023-11-28T11:38:55.451 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2023-11-28T11:38:55.451 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.6" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T11:38:55.514 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T11:38:55.515 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T11:38:55.515 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:38:55.515 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T11:38:55.515 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T11:38:55.515 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T11:38:55.515 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T11:38:55.515 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-11-28T11:38:55.540 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T11:38:55.549 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T11:38:55.556 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T11:38:55.557 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2023-11-28T11:38:55.559 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2023-11-28T11:38:55.577 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T11:38:55.584 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T11:38:55.584 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2023-11-28T11:38:55.617 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2023-11-28T11:38:55.650 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T11:38:55.713 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T11:38:55.734 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T11:38:55.734 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-28T11:38:55.734 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-28T11:38:55.735 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T11:38:55.735 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T11:38:55.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:38:55.753 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2023-11-28T11:38:55.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:38:55.754 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2023-11-28T11:38:55.814 INFO:teuthology.orchestra.run.smithi144.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T11:38:55.814 INFO:teuthology.task.kernel:Running kernel on smithi144: 4.18.0-372.9.1.el8.x86_64 2023-11-28T11:38:55.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2023-11-28T11:38:55.815 INFO:teuthology.orchestra.run.smithi177.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T11:38:55.816 INFO:teuthology.task.kernel:Running kernel on smithi177: 4.18.0-372.9.1.el8.x86_64 2023-11-28T11:38:55.816 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2023-11-28T11:38:56.933 INFO:teuthology.orchestra.run.smithi144.stdout:Updating Subscription Management repositories. 2023-11-28T11:38:56.933 INFO:teuthology.orchestra.run.smithi144.stdout:Unable to read consumer identity 2023-11-28T11:38:56.953 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T11:38:56.953 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-11-28T11:38:58.371 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-28T11:38:58.374 INFO:teuthology.orchestra.run.smithi144.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-28T11:39:00.125 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 16 MB 00:01 2023-11-28T11:39:00.188 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 16 MB 00:01 2023-11-28T11:39:03.658 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 401 kB/s | 24 kB 00:00 2023-11-28T11:39:03.663 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 402 kB/s | 24 kB 00:00 2023-11-28T11:39:04.324 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T11:39:04.352 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T11:39:04.385 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-28T11:39:04.386 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2023-11-28T11:39:04.386 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-28T11:39:04.404 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2023-11-28T11:39:04.405 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2023-11-28T11:39:04.405 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2023-11-28T11:39:04.497 DEBUG:teuthology.orchestra.run.smithi177:> echo no | sudo yum reinstall kernel || true 2023-11-28T11:39:04.526 DEBUG:teuthology.orchestra.run.smithi144:> echo no | sudo yum reinstall kernel || true 2023-11-28T11:39:04.826 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T11:39:04.827 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-11-28T11:39:04.857 INFO:teuthology.orchestra.run.smithi144.stdout:Updating Subscription Management repositories. 2023-11-28T11:39:04.857 INFO:teuthology.orchestra.run.smithi144.stdout:Unable to read consumer identity 2023-11-28T11:39:05.094 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 11:39:03 AM UTC. 2023-11-28T11:39:05.121 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 11:39:03 AM UTC. 2023-11-28T11:39:05.137 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T11:39:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T11:39:05.187 INFO:teuthology.orchestra.run.smithi177.stderr:Error: No packages marked for reinstall. 2023-11-28T11:39:05.197 INFO:teuthology.orchestra.run.smithi144.stderr:Error: No packages marked for reinstall. 2023-11-28T11:39:05.234 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum reinstall -y kernel || true 2023-11-28T11:39:05.244 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum reinstall -y kernel || true 2023-11-28T11:39:05.564 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T11:39:05.565 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-11-28T11:39:05.575 INFO:teuthology.orchestra.run.smithi144.stdout:Updating Subscription Management repositories. 2023-11-28T11:39:05.575 INFO:teuthology.orchestra.run.smithi144.stdout:Unable to read consumer identity 2023-11-28T11:39:05.825 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 11:39:03 AM UTC. 2023-11-28T11:39:05.835 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 11:39:03 AM UTC. 2023-11-28T11:39:05.865 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T11:39:05.874 INFO:teuthology.orchestra.run.smithi144.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T11:39:05.903 INFO:teuthology.orchestra.run.smithi144.stderr:Error: No packages marked for reinstall. 2023-11-28T11:39:05.903 INFO:teuthology.orchestra.run.smithi177.stderr:Error: No packages marked for reinstall. 2023-11-28T11:39:05.950 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T11:39:05.952 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T11:39:06.032 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T11:39:06.032 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T11:39:06.032 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T11:39:06.033 DEBUG:teuthology.parallel:result is None 2023-11-28T11:39:06.033 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T11:39:06.034 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T11:39:06.034 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T11:39:06.034 DEBUG:teuthology.parallel:result is None 2023-11-28T11:39:06.034 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T11:39:06.042 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T11:39:06.042 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:39:06.044 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:39:06.060 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T11:39:06.068 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T11:39:06.077 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T11:39:06.078 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:39:06.101 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:39:06.124 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T11:39:06.132 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T11:39:06.132 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:39:06.159 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:39:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:39:06.200 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:39:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:39:06.218 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:39:06.218 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T11:39:06.225 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T11:39:06.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:39:06.260 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:39:06.287 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T11:39:06.296 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T11:39:06.297 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:39:06.327 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:39:06.344 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:39:06.422 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:39:06.472 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:39:06.552 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:39:06.596 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:39:06.597 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:39:06.664 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:39:06.711 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:39:06.761 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:39:06.848 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:39:06.888 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:39:06.889 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:39:06.958 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2023-11-28T11:39:06.961 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2023-11-28T11:39:07.016 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:39:07.056 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:39:07.467 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T11:39:07.476 INFO:teuthology.task.internal:Starting timer... 2023-11-28T11:39:07.477 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T11:39:07.496 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T11:39:07.512 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T11:39:07.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2023-11-28T11:39:07.574 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2023-11-28T11:39:07.576 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2023-11-28T11:39:07.638 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2023-11-28T11:39:07.640 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T11:39:07.640 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2023-11-28T11:39:07.673 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2023-11-28T11:39:07.673 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2023-11-28T11:39:07.699 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2023-11-28T11:39:07.699 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi144.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2023-11-28T11:39:07.699 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:39:07.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:39:07.740 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:39:07.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:39:07.768 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T11:39:07.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:39:07.807 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:39:07.835 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T11:39:07.844 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-28T11:39:07.850 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-28T11:39:07.967 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T11:39:07.986 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T11:39:07.988 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi144.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:48:08.690 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2023-11-28T11:48:08.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-11-28T11:48:08.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:48:08.761 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-11-28T11:48:08.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-11-28T11:48:08.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-11-28T11:48:08.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:48:08.896 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-11-28T11:48:08.964 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-11-28T11:48:08.965 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:48:08.976 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:48:08.977 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:48:08.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:48:08.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:48:08.980 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:48:09.009 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:48:09.024 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:48:09.038 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:48:09.053 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:48:09.110 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2023-11-28T11:48:09.122 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2023-11-28T11:48:09.123 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2023-11-28T11:48:09.135 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2023-11-28T11:48:09.137 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:48:09.149 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:48:09.151 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:48:09.163 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:48:09.223 INFO:teuthology.orchestra.run.smithi144.stderr:bash: ntpq: command not found 2023-11-28T11:48:09.227 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:48:09.227 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2023-11-28T11:48:09.227 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.227 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.227 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.227 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.242 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-11-28T11:48:09.246 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:48:09.246 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-11-28T11:48:09.246 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.246 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.246 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.246 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:48:09.247 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:48:09.257 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:48:09.257 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:48:09.257 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:48:09.258 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:48:09.263 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:48:09.263 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:48:09.264 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:48:09.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:48:09.266 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:48:09.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:48:09.486 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:48:09.486 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:48:09.500 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:48:09.500 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:48:09.613 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:48:09.614 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:48:09.614 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:48:09.622 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:48:09.623 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:48:09.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:48:09.643 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:48:09.643 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:48:09.650 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:48:09.650 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:48:09.713 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:48:09.718 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:48:09.793 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:48:09.794 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:48:09.859 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2023-11-28T11:48:09.860 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2023-11-28T11:48:09.861 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2023-11-28T11:48:09.862 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2023-11-28T11:48:10.232 INFO:teuthology.orchestra.run.smithi144.stdout:Updating Subscription Management repositories. 2023-11-28T11:48:10.234 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T11:48:10.510 INFO:teuthology.orchestra.run.smithi144.stdout:44 files removed 2023-11-28T11:48:10.517 INFO:teuthology.orchestra.run.smithi177.stdout:44 files removed 2023-11-28T11:48:10.558 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:48:10.564 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:48:10.887 INFO:teuthology.orchestra.run.smithi144.stdout:Updating Subscription Management repositories. 2023-11-28T11:48:10.893 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-28T11:48:11.617 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 217 kB/s | 80 kB 00:00 2023-11-28T11:48:11.618 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2023-11-28T11:48:11.969 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-11-28T11:48:11.984 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-11-28T11:48:12.227 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-11-28T11:48:12.245 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2023-11-28T11:48:12.378 INFO:teuthology.orchestra.run.smithi144.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-28T11:48:12.378 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-11-28T11:48:13.640 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-11-28T11:48:14.356 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 16 MB 00:01 2023-11-28T11:48:17.196 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 583 kB/s | 24 kB 00:00 2023-11-28T11:48:18.049 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-11-28T11:48:18.095 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 80 kB/s | 24 kB 00:00 2023-11-28T11:48:19.005 INFO:teuthology.orchestra.run.smithi144.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-11-28T11:48:25.883 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-11-28T11:48:27.098 INFO:teuthology.orchestra.run.smithi144.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-11-28T11:48:37.105 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:48:37.106 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:48:37.121 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:48:37.122 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:48:37.124 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:48:37.125 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:48:37.127 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:48:37.130 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:48:37.132 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:48:37.266 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-28T11:48:37.278 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:48:37.279 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:48:37.280 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:48:37.281 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:48:37.282 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:48:37.283 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:48:37.284 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:48:37.285 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:48:37.286 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:48:37.286 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:48:37.286 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:48:37.286 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:48:37.286 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:48:37.286 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:48:37.287 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:48:37.287 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:48:37.287 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:48:37.287 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:48:37.287 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:48:37.288 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:48:37.288 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:48:37.288 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:48:37.288 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:48:37.288 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:48:37.288 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:48:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:48:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:48:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:48:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:48:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:48:37.289 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:48:37.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:48:37.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:48:37.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:48:37.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:48:37.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:48:37.290 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:48:37.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:48:37.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:48:37.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:48:37.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:48:37.291 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:48:37.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:48:37.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:48:37.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:48:37.292 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:48:37.293 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:48:37.294 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:48:37.295 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout:Enabling module streams: 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql 10 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout:Install 128 Packages 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 3 Packages 2023-11-28T11:48:37.296 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:48:37.297 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 203 M 2023-11-28T11:48:37.297 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-11-28T11:48:37.448 INFO:teuthology.orchestra.run.smithi177.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 57 kB/s | 6.4 kB 00:00 2023-11-28T11:48:37.623 INFO:teuthology.orchestra.run.smithi177.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 4.7 MB/s | 845 kB 00:00 2023-11-28T11:48:37.674 INFO:teuthology.orchestra.run.smithi177.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 3.2 MB/s | 165 kB 00:00 2023-11-28T11:48:37.940 INFO:teuthology.orchestra.run.smithi177.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 8.5 MB/s | 2.3 MB 00:00 2023-11-28T11:48:38.099 INFO:teuthology.orchestra.run.smithi177.stdout:(5/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 7.0 MB/s | 5.4 MB 00:00 2023-11-28T11:48:38.137 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:48:38.138 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:48:38.141 INFO:teuthology.orchestra.run.smithi177.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 7.7 MB/s | 1.5 MB 00:00 2023-11-28T11:48:38.153 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:48:38.154 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:48:38.158 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:48:38.158 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:48:38.158 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:48:38.162 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:48:38.163 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:48:38.294 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2023-11-28T11:48:38.306 INFO:teuthology.orchestra.run.smithi144.stdout:=========================================================================================================================== 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout:=========================================================================================================================== 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:48:38.307 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:48:38.308 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:48:38.309 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:48:38.310 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:48:38.311 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:48:38.312 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:48:38.313 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:48:38.314 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:48:38.314 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:48:38.314 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:48:38.314 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:48:38.314 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:48:38.314 INFO:teuthology.orchestra.run.smithi144.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:48:38.315 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:48:38.315 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:48:38.315 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:48:38.315 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:48:38.315 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:48:38.316 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:48:38.316 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:48:38.316 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:48:38.316 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:48:38.316 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:48:38.316 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:48:38.317 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:48:38.317 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:48:38.317 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:48:38.317 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:48:38.317 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:48:38.317 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:48:38.318 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:48:38.319 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:48:38.320 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:48:38.321 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:Enabling module streams: 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql 10 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:=========================================================================================================================== 2023-11-28T11:48:38.322 INFO:teuthology.orchestra.run.smithi144.stdout:Install 128 Packages 2023-11-28T11:48:38.323 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 3 Packages 2023-11-28T11:48:38.323 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:48:38.323 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 203 M 2023-11-28T11:48:38.324 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2023-11-28T11:48:38.508 INFO:teuthology.orchestra.run.smithi177.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 11 MB/s | 4.6 MB 00:00 2023-11-28T11:48:38.547 INFO:teuthology.orchestra.run.smithi144.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 38 kB/s | 6.4 kB 00:00 2023-11-28T11:48:39.531 INFO:teuthology.orchestra.run.smithi144.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 859 kB/s | 845 kB 00:00 2023-11-28T11:48:39.848 INFO:teuthology.orchestra.run.smithi144.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 522 kB/s | 165 kB 00:00 2023-11-28T11:48:39.900 INFO:teuthology.orchestra.run.smithi177.stdout:(8/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el8 9.4 MB/s | 13 MB 00:01 2023-11-28T11:48:39.967 INFO:teuthology.orchestra.run.smithi177.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 363 kB/s | 24 kB 00:00 2023-11-28T11:48:40.393 INFO:teuthology.orchestra.run.smithi177.stdout:(10/131): ceph-common-18.0.0-2697.g2f5f6028.el8 7.0 MB/s | 22 MB 00:03 2023-11-28T11:48:40.593 INFO:teuthology.orchestra.run.smithi177.stdout:(11/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 7.2 MB/s | 18 MB 00:02 2023-11-28T11:48:40.618 INFO:teuthology.orchestra.run.smithi177.stdout:(12/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.1 MB/s | 257 kB 00:00 2023-11-28T11:48:40.643 INFO:teuthology.orchestra.run.smithi177.stdout:(13/131): libcephfs-devel-18.0.0-2697.g2f5f6028 610 kB/s | 30 kB 00:00 2023-11-28T11:48:40.665 INFO:teuthology.orchestra.run.smithi144.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 2.8 MB/s | 2.3 MB 00:00 2023-11-28T11:48:40.693 INFO:teuthology.orchestra.run.smithi177.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.7 MB/s | 137 kB 00:00 2023-11-28T11:48:40.760 INFO:teuthology.orchestra.run.smithi177.stdout:(15/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 5.3 MB/s | 767 kB 00:00 2023-11-28T11:48:40.786 INFO:teuthology.orchestra.run.smithi177.stdout:(16/131): librados-devel-18.0.0-2697.g2f5f6028. 1.5 MB/s | 139 kB 00:00 2023-11-28T11:48:40.861 INFO:teuthology.orchestra.run.smithi177.stdout:(17/131): libradosstriper1-18.0.0-2697.g2f5f602 4.6 MB/s | 474 kB 00:00 2023-11-28T11:48:40.945 INFO:teuthology.orchestra.run.smithi177.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 536 kB/s | 45 kB 00:00 2023-11-28T11:48:41.028 INFO:teuthology.orchestra.run.smithi177.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.4 MB/s | 117 kB 00:00 2023-11-28T11:48:41.082 INFO:teuthology.orchestra.run.smithi144.stdout:(5/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-11-28T11:48:41.112 INFO:teuthology.orchestra.run.smithi177.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.2 MB/s | 190 kB 00:00 2023-11-28T11:48:41.204 INFO:teuthology.orchestra.run.smithi177.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 4.0 MB/s | 371 kB 00:00 2023-11-28T11:48:41.312 INFO:teuthology.orchestra.run.smithi177.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 3.3 MB/s | 364 kB 00:00 2023-11-28T11:48:41.357 INFO:teuthology.orchestra.run.smithi144.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 2.2 MB/s | 1.5 MB 00:00 2023-11-28T11:48:41.396 INFO:teuthology.orchestra.run.smithi177.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.3 MB/s | 107 kB 00:00 2023-11-28T11:48:41.480 INFO:teuthology.orchestra.run.smithi177.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.1 MB/s | 94 kB 00:00 2023-11-28T11:48:41.588 INFO:teuthology.orchestra.run.smithi177.stdout:(25/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 8.1 MB/s | 6.5 MB 00:00 2023-11-28T11:48:41.672 INFO:teuthology.orchestra.run.smithi177.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 2.1 MB/s | 180 kB 00:00 2023-11-28T11:48:41.755 INFO:teuthology.orchestra.run.smithi177.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 277 kB/s | 23 kB 00:00 2023-11-28T11:48:41.831 INFO:teuthology.orchestra.run.smithi177.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.6 MB/s | 123 kB 00:00 2023-11-28T11:48:42.006 INFO:teuthology.orchestra.run.smithi177.stdout:(29/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 6.6 MB/s | 3.4 MB 00:00 2023-11-28T11:48:42.140 INFO:teuthology.orchestra.run.smithi177.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 5.6 MB/s | 1.7 MB 00:00 2023-11-28T11:48:42.198 INFO:teuthology.orchestra.run.smithi177.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 4.0 MB/s | 236 kB 00:00 2023-11-28T11:48:42.248 INFO:teuthology.orchestra.run.smithi177.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 947 kB/s | 47 kB 00:00 2023-11-28T11:48:42.299 INFO:teuthology.orchestra.run.smithi177.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 290 kB/s | 15 kB 00:00 2023-11-28T11:48:42.365 INFO:teuthology.orchestra.run.smithi177.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 3.9 MB/s | 263 kB 00:00 2023-11-28T11:48:42.424 INFO:teuthology.orchestra.run.smithi177.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.4 MB/s | 83 kB 00:00 2023-11-28T11:48:42.483 INFO:teuthology.orchestra.run.smithi177.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 307 kB/s | 18 kB 00:00 2023-11-28T11:48:42.675 INFO:teuthology.orchestra.run.smithi177.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-11-28T11:48:42.682 INFO:teuthology.orchestra.run.smithi144.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 2.9 MB/s | 4.6 MB 00:01 2023-11-28T11:48:42.717 INFO:teuthology.orchestra.run.smithi177.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-11-28T11:48:42.751 INFO:teuthology.orchestra.run.smithi177.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2023-11-28T11:48:42.810 INFO:teuthology.orchestra.run.smithi177.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 8.0 MB/s | 476 kB 00:00 2023-11-28T11:48:42.852 INFO:teuthology.orchestra.run.smithi177.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-11-28T11:48:42.894 INFO:teuthology.orchestra.run.smithi177.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 4.3 MB/s | 185 kB 00:00 2023-11-28T11:48:42.936 INFO:teuthology.orchestra.run.smithi177.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 791 kB/s | 33 kB 00:00 2023-11-28T11:48:42.977 INFO:teuthology.orchestra.run.smithi177.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 392 kB/s | 16 kB 00:00 2023-11-28T11:48:43.019 INFO:teuthology.orchestra.run.smithi177.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 4.0 MB/s | 173 kB 00:00 2023-11-28T11:48:43.061 INFO:teuthology.orchestra.run.smithi177.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 9.0 MB/s | 384 kB 00:00 2023-11-28T11:48:43.103 INFO:teuthology.orchestra.run.smithi177.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-11-28T11:48:43.195 INFO:teuthology.orchestra.run.smithi177.stdout:(48/131): ceph-mgr-diskprediction-local-18.0.0- 6.2 MB/s | 7.4 MB 00:01 2023-11-28T11:48:43.220 INFO:teuthology.orchestra.run.smithi177.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-11-28T11:48:43.246 INFO:teuthology.orchestra.run.smithi177.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 349 kB/s | 18 kB 00:00 2023-11-28T11:48:43.288 INFO:teuthology.orchestra.run.smithi177.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-11-28T11:48:43.329 INFO:teuthology.orchestra.run.smithi177.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-11-28T11:48:43.371 INFO:teuthology.orchestra.run.smithi177.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-11-28T11:48:43.438 INFO:teuthology.orchestra.run.smithi177.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 5.3 MB/s | 1.2 MB 00:00 2023-11-28T11:48:43.464 INFO:teuthology.orchestra.run.smithi177.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2023-11-28T11:48:43.489 INFO:teuthology.orchestra.run.smithi177.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-11-28T11:48:43.514 INFO:teuthology.orchestra.run.smithi177.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 668 kB/s | 34 kB 00:00 2023-11-28T11:48:43.540 INFO:teuthology.orchestra.run.smithi177.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-11-28T11:48:43.565 INFO:teuthology.orchestra.run.smithi177.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-11-28T11:48:43.590 INFO:teuthology.orchestra.run.smithi177.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-11-28T11:48:43.616 INFO:teuthology.orchestra.run.smithi177.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-11-28T11:48:43.641 INFO:teuthology.orchestra.run.smithi177.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 580 kB/s | 29 kB 00:00 2023-11-28T11:48:43.666 INFO:teuthology.orchestra.run.smithi177.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 546 kB/s | 27 kB 00:00 2023-11-28T11:48:43.691 INFO:teuthology.orchestra.run.smithi177.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-11-28T11:48:43.725 INFO:teuthology.orchestra.run.smithi177.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-11-28T11:48:43.750 INFO:teuthology.orchestra.run.smithi177.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-28T11:48:43.776 INFO:teuthology.orchestra.run.smithi177.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-28T11:48:43.801 INFO:teuthology.orchestra.run.smithi177.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2023-11-28T11:48:43.826 INFO:teuthology.orchestra.run.smithi177.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:48:43.885 INFO:teuthology.orchestra.run.smithi177.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2023-11-28T11:48:43.952 INFO:teuthology.orchestra.run.smithi177.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-11-28T11:48:44.002 INFO:teuthology.orchestra.run.smithi177.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-11-28T11:48:44.052 INFO:teuthology.orchestra.run.smithi177.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-11-28T11:48:44.103 INFO:teuthology.orchestra.run.smithi177.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-28T11:48:44.253 INFO:teuthology.orchestra.run.smithi177.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 32 MB/s | 14 MB 00:00 2023-11-28T11:48:44.678 INFO:teuthology.orchestra.run.smithi177.stdout:(76/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 9.9 MB/s | 47 MB 00:04 2023-11-28T11:48:44.703 INFO:teuthology.orchestra.run.smithi177.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 65 kB/s | 39 kB 00:00 2023-11-28T11:48:44.745 INFO:teuthology.orchestra.run.smithi177.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-11-28T11:48:44.771 INFO:teuthology.orchestra.run.smithi177.stdout:(79/131): bison-3.0.4-10.el8.x86_64.rpm 1.3 MB/s | 688 kB 00:00 2023-11-28T11:48:44.791 INFO:teuthology.orchestra.run.smithi144.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 3.4 MB/s | 22 MB 00:06 2023-11-28T11:48:44.796 INFO:teuthology.orchestra.run.smithi177.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-11-28T11:48:44.821 INFO:teuthology.orchestra.run.smithi177.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2023-11-28T11:48:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 490 kB/s | 24 kB 00:00 2023-11-28T11:48:44.855 INFO:teuthology.orchestra.run.smithi177.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-11-28T11:48:44.871 INFO:teuthology.orchestra.run.smithi177.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 530 kB/s | 27 kB 00:00 2023-11-28T11:48:44.905 INFO:teuthology.orchestra.run.smithi177.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-11-28T11:48:44.922 INFO:teuthology.orchestra.run.smithi177.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 875 kB/s | 44 kB 00:00 2023-11-28T11:48:44.981 INFO:teuthology.orchestra.run.smithi177.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-11-28T11:48:45.076 INFO:teuthology.orchestra.run.smithi177.stdout:(87/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-11-28T11:48:45.106 INFO:teuthology.orchestra.run.smithi177.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.8 MB/s | 225 kB 00:00 2023-11-28T11:48:45.150 INFO:teuthology.orchestra.run.smithi144.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 5.3 MB/s | 13 MB 00:02 2023-11-28T11:48:45.207 INFO:teuthology.orchestra.run.smithi177.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 491 kB/s | 64 kB 00:00 2023-11-28T11:48:45.217 INFO:teuthology.orchestra.run.smithi144.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 3.8 MB/s | 257 kB 00:00 2023-11-28T11:48:45.232 INFO:teuthology.orchestra.run.smithi177.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-11-28T11:48:45.257 INFO:teuthology.orchestra.run.smithi177.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-11-28T11:48:45.267 INFO:teuthology.orchestra.run.smithi144.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 612 kB/s | 30 kB 00:00 2023-11-28T11:48:45.282 INFO:teuthology.orchestra.run.smithi177.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-28T11:48:45.341 INFO:teuthology.orchestra.run.smithi177.stdout:(93/131): python3-numpy-1.14.3-10.el8.x86_64.rp 8.6 MB/s | 3.7 MB 00:00 2023-11-28T11:48:45.375 INFO:teuthology.orchestra.run.smithi177.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.9 MB/s | 709 kB 00:00 2023-11-28T11:48:45.400 INFO:teuthology.orchestra.run.smithi177.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-11-28T11:48:45.450 INFO:teuthology.orchestra.run.smithi177.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-11-28T11:48:45.460 INFO:teuthology.orchestra.run.smithi144.stdout:(13/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 4.3 MB/s | 18 MB 00:04 2023-11-28T11:48:45.492 INFO:teuthology.orchestra.run.smithi144.stdout:(14/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 3.3 MB/s | 767 kB 00:00 2023-11-28T11:48:45.501 INFO:teuthology.orchestra.run.smithi177.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-11-28T11:48:45.518 INFO:teuthology.orchestra.run.smithi144.stdout:(15/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.3 MB/s | 137 kB 00:00 2023-11-28T11:48:45.543 INFO:teuthology.orchestra.run.smithi144.stdout:(16/131): librados-devel-18.0.0-2697.g2f5f6028. 2.7 MB/s | 139 kB 00:00 2023-11-28T11:48:45.568 INFO:teuthology.orchestra.run.smithi177.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-11-28T11:48:45.610 INFO:teuthology.orchestra.run.smithi144.stdout:(17/131): libradosstriper1-18.0.0-2697.g2f5f602 5.0 MB/s | 474 kB 00:00 2023-11-28T11:48:45.635 INFO:teuthology.orchestra.run.smithi177.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-11-28T11:48:45.652 INFO:teuthology.orchestra.run.smithi144.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 1.1 MB/s | 45 kB 00:00 2023-11-28T11:48:45.685 INFO:teuthology.orchestra.run.smithi177.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-11-28T11:48:45.694 INFO:teuthology.orchestra.run.smithi144.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 2.7 MB/s | 117 kB 00:00 2023-11-28T11:48:45.744 INFO:teuthology.orchestra.run.smithi177.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-11-28T11:48:45.752 INFO:teuthology.orchestra.run.smithi144.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 3.2 MB/s | 190 kB 00:00 2023-11-28T11:48:45.769 INFO:teuthology.orchestra.run.smithi177.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 560 kB/s | 47 kB 00:00 2023-11-28T11:48:45.794 INFO:teuthology.orchestra.run.smithi177.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-11-28T11:48:45.819 INFO:teuthology.orchestra.run.smithi177.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-11-28T11:48:45.828 INFO:teuthology.orchestra.run.smithi144.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 4.8 MB/s | 371 kB 00:00 2023-11-28T11:48:45.844 INFO:teuthology.orchestra.run.smithi177.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-28T11:48:45.870 INFO:teuthology.orchestra.run.smithi177.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-11-28T11:48:45.895 INFO:teuthology.orchestra.run.smithi177.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-28T11:48:45.903 INFO:teuthology.orchestra.run.smithi144.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 4.7 MB/s | 364 kB 00:00 2023-11-28T11:48:45.920 INFO:teuthology.orchestra.run.smithi177.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-11-28T11:48:45.946 INFO:teuthology.orchestra.run.smithi177.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-28T11:48:45.953 INFO:teuthology.orchestra.run.smithi144.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 107 kB 00:00 2023-11-28T11:48:45.971 INFO:teuthology.orchestra.run.smithi177.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-11-28T11:48:46.004 INFO:teuthology.orchestra.run.smithi144.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-28T11:48:46.004 INFO:teuthology.orchestra.run.smithi177.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-11-28T11:48:46.030 INFO:teuthology.orchestra.run.smithi177.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 320 kB/s | 19 kB 00:00 2023-11-28T11:48:46.055 INFO:teuthology.orchestra.run.smithi177.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-28T11:48:46.089 INFO:teuthology.orchestra.run.smithi177.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-11-28T11:48:46.157 INFO:teuthology.orchestra.run.smithi177.stdout:(115/131): postgresql-server-10.21-2.module+el8 7.7 MB/s | 5.1 MB 00:00 2023-11-28T11:48:46.181 INFO:teuthology.orchestra.run.smithi177.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 142 kB/s | 18 kB 00:00 2023-11-28T11:48:46.206 INFO:teuthology.orchestra.run.smithi177.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-11-28T11:48:46.240 INFO:teuthology.orchestra.run.smithi177.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-11-28T11:48:46.257 INFO:teuthology.orchestra.run.smithi177.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-11-28T11:48:46.290 INFO:teuthology.orchestra.run.smithi177.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-28T11:48:46.307 INFO:teuthology.orchestra.run.smithi177.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-28T11:48:46.341 INFO:teuthology.orchestra.run.smithi177.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-28T11:48:46.358 INFO:teuthology.orchestra.run.smithi177.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-11-28T11:48:46.396 INFO:teuthology.orchestra.run.smithi144.stdout:(25/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 7.6 MB/s | 6.5 MB 00:00 2023-11-28T11:48:46.408 INFO:teuthology.orchestra.run.smithi177.stdout:(124/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 10 MB/s | 2.3 MB 00:00 2023-11-28T11:48:46.425 INFO:teuthology.orchestra.run.smithi177.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 460 kB/s | 39 kB 00:00 2023-11-28T11:48:46.446 INFO:teuthology.orchestra.run.smithi144.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 3.5 MB/s | 180 kB 00:00 2023-11-28T11:48:46.467 INFO:teuthology.orchestra.run.smithi177.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 5.8 MB/s | 644 kB 00:00 2023-11-28T11:48:46.484 INFO:teuthology.orchestra.run.smithi177.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-11-28T11:48:46.505 INFO:teuthology.orchestra.run.smithi144.stdout:(27/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 6.9 MB/s | 3.4 MB 00:00 2023-11-28T11:48:46.510 INFO:teuthology.orchestra.run.smithi177.stdout:(128/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.7 MB/s | 171 kB 00:00 2023-11-28T11:48:46.530 INFO:teuthology.orchestra.run.smithi144.stdout:(28/131): ceph-grafana-dashboards-18.0.0-2697.g 276 kB/s | 23 kB 00:00 2023-11-28T11:48:46.556 INFO:teuthology.orchestra.run.smithi144.stdout:(29/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 2.4 MB/s | 123 kB 00:00 2023-11-28T11:48:46.568 INFO:teuthology.orchestra.run.smithi177.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-28T11:48:46.881 INFO:teuthology.orchestra.run.smithi144.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 4.9 MB/s | 1.7 MB 00:00 2023-11-28T11:48:47.023 INFO:teuthology.orchestra.run.smithi144.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 1.6 MB/s | 236 kB 00:00 2023-11-28T11:48:47.073 INFO:teuthology.orchestra.run.smithi144.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 946 kB/s | 47 kB 00:00 2023-11-28T11:48:47.077 INFO:teuthology.orchestra.run.smithi177.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 6.0 MB/s | 3.7 MB 00:00 2023-11-28T11:48:47.124 INFO:teuthology.orchestra.run.smithi144.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 291 kB/s | 15 kB 00:00 2023-11-28T11:48:47.135 INFO:teuthology.orchestra.run.smithi177.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 5.5 MB/s | 3.6 MB 00:00 2023-11-28T11:48:47.137 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:48:47.138 INFO:teuthology.orchestra.run.smithi177.stdout:Total 21 MB/s | 203 MB 00:09 2023-11-28T11:48:47.199 INFO:teuthology.orchestra.run.smithi144.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 3.4 MB/s | 263 kB 00:00 2023-11-28T11:48:47.249 INFO:teuthology.orchestra.run.smithi144.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.6 MB/s | 83 kB 00:00 2023-11-28T11:48:47.299 INFO:teuthology.orchestra.run.smithi144.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 362 kB/s | 18 kB 00:00 2023-11-28T11:48:47.675 INFO:teuthology.orchestra.run.smithi144.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.6 MB/s | 7.4 MB 00:01 2023-11-28T11:48:47.850 INFO:teuthology.orchestra.run.smithi144.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 556 kB/s | 306 kB 00:00 2023-11-28T11:48:47.958 INFO:teuthology.orchestra.run.smithi144.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 698 kB/s | 75 kB 00:00 2023-11-28T11:48:47.980 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-11-28T11:48:48.084 INFO:teuthology.orchestra.run.smithi144.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 144 kB/s | 59 kB 00:00 2023-11-28T11:48:48.101 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-11-28T11:48:48.101 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-11-28T11:48:48.134 INFO:teuthology.orchestra.run.smithi144.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-11-28T11:48:48.201 INFO:teuthology.orchestra.run.smithi144.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-11-28T11:48:48.243 INFO:teuthology.orchestra.run.smithi144.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-11-28T11:48:48.310 INFO:teuthology.orchestra.run.smithi144.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 303 kB/s | 33 kB 00:00 2023-11-28T11:48:48.335 INFO:teuthology.orchestra.run.smithi144.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2023-11-28T11:48:48.452 INFO:teuthology.orchestra.run.smithi144.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-11-28T11:48:48.502 INFO:teuthology.orchestra.run.smithi144.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 898 kB/s | 173 kB 00:00 2023-11-28T11:48:48.553 INFO:teuthology.orchestra.run.smithi144.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 812 kB/s | 81 kB 00:00 2023-11-28T11:48:48.595 INFO:teuthology.orchestra.run.smithi144.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-11-28T11:48:48.645 INFO:teuthology.orchestra.run.smithi144.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-11-28T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2023-11-28T11:48:48.821 INFO:teuthology.orchestra.run.smithi144.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 704 kB/s | 59 kB 00:00 2023-11-28T11:48:48.888 INFO:teuthology.orchestra.run.smithi144.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 4.0 MB/s | 1.2 MB 00:00 2023-11-28T11:48:48.905 INFO:teuthology.orchestra.run.smithi144.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 720 kB/s | 60 kB 00:00 2023-11-28T11:48:49.022 INFO:teuthology.orchestra.run.smithi144.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 134 kB/s | 16 kB 00:00 2023-11-28T11:48:49.047 INFO:teuthology.orchestra.run.smithi144.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-11-28T11:48:49.220 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-11-28T11:48:49.224 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-11-28T11:48:49.314 INFO:teuthology.orchestra.run.smithi144.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 115 kB/s | 34 kB 00:00 2023-11-28T11:48:49.339 INFO:teuthology.orchestra.run.smithi144.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 673 kB/s | 196 kB 00:00 2023-11-28T11:48:49.423 INFO:teuthology.orchestra.run.smithi144.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2023-11-28T11:48:49.448 INFO:teuthology.orchestra.run.smithi144.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-11-28T11:48:49.516 INFO:teuthology.orchestra.run.smithi144.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2023-11-28T11:48:49.541 INFO:teuthology.orchestra.run.smithi144.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-11-28T11:48:49.608 INFO:teuthology.orchestra.run.smithi144.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-11-28T11:48:49.641 INFO:teuthology.orchestra.run.smithi144.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-11-28T11:48:49.725 INFO:teuthology.orchestra.run.smithi144.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-11-28T11:48:49.750 INFO:teuthology.orchestra.run.smithi144.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 563 kB/s | 61 kB 00:00 2023-11-28T11:48:49.826 INFO:teuthology.orchestra.run.smithi144.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 852 kB/s | 85 kB 00:00 2023-11-28T11:48:49.851 INFO:teuthology.orchestra.run.smithi144.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-11-28T11:48:49.876 INFO:teuthology.orchestra.run.smithi144.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:48:49.935 INFO:teuthology.orchestra.run.smithi144.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-11-28T11:48:49.977 INFO:teuthology.orchestra.run.smithi144.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-11-28T11:48:50.410 INFO:teuthology.orchestra.run.smithi144.stdout:(72/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 8.4 MB/s | 47 MB 00:05 2023-11-28T11:48:50.435 INFO:teuthology.orchestra.run.smithi144.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 93 kB/s | 43 kB 00:00 2023-11-28T11:48:50.486 INFO:teuthology.orchestra.run.smithi144.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 2.6 MB/s | 1.7 MB 00:00 2023-11-28T11:48:50.511 INFO:teuthology.orchestra.run.smithi144.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 691 kB/s | 48 kB 00:00 2023-11-28T11:48:50.545 INFO:teuthology.orchestra.run.smithi144.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2023-11-28T11:48:50.570 INFO:teuthology.orchestra.run.smithi144.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 805 kB/s | 43 kB 00:00 2023-11-28T11:48:50.595 INFO:teuthology.orchestra.run.smithi144.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 798 kB/s | 87 kB 00:00 2023-11-28T11:48:50.621 INFO:teuthology.orchestra.run.smithi144.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-11-28T11:48:50.646 INFO:teuthology.orchestra.run.smithi144.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-11-28T11:48:50.671 INFO:teuthology.orchestra.run.smithi144.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-11-28T11:48:50.697 INFO:teuthology.orchestra.run.smithi144.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-28T11:48:50.731 INFO:teuthology.orchestra.run.smithi144.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-11-28T11:48:50.755 INFO:teuthology.orchestra.run.smithi144.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-11-28T11:48:50.789 INFO:teuthology.orchestra.run.smithi144.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.7 MB/s | 559 kB 00:00 2023-11-28T11:48:50.814 INFO:teuthology.orchestra.run.smithi144.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-28T11:48:50.840 INFO:teuthology.orchestra.run.smithi144.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-11-28T11:48:50.873 INFO:teuthology.orchestra.run.smithi144.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-11-28T11:48:50.924 INFO:teuthology.orchestra.run.smithi144.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 990 kB/s | 83 kB 00:00 2023-11-28T11:48:50.940 INFO:teuthology.orchestra.run.smithi144.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 897 kB/s | 60 kB 00:00 2023-11-28T11:48:50.966 INFO:teuthology.orchestra.run.smithi144.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-28T11:48:50.991 INFO:teuthology.orchestra.run.smithi144.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-28T11:48:51.008 INFO:teuthology.orchestra.run.smithi144.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 918 kB/s | 39 kB 00:00 2023-11-28T11:48:51.050 INFO:teuthology.orchestra.run.smithi144.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-28T11:48:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-11-28T11:48:51.101 INFO:teuthology.orchestra.run.smithi144.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-28T11:48:51.168 INFO:teuthology.orchestra.run.smithi144.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.1 MB/s | 2.3 MB 00:00 2023-11-28T11:48:51.193 INFO:teuthology.orchestra.run.smithi144.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-11-28T11:48:51.218 INFO:teuthology.orchestra.run.smithi144.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-11-28T11:48:51.252 INFO:teuthology.orchestra.run.smithi144.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-11-28T11:48:51.277 INFO:teuthology.orchestra.run.smithi144.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-11-28T11:48:51.302 INFO:teuthology.orchestra.run.smithi144.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 775 kB/s | 39 kB 00:00 2023-11-28T11:48:51.344 INFO:teuthology.orchestra.run.smithi144.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-11-28T11:48:51.378 INFO:teuthology.orchestra.run.smithi144.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-11-28T11:48:51.397 INFO:teuthology.orchestra.run.smithi144.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-11-28T11:48:51.420 INFO:teuthology.orchestra.run.smithi144.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-28T11:48:51.470 INFO:teuthology.orchestra.run.smithi144.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-11-28T11:48:51.504 INFO:teuthology.orchestra.run.smithi144.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 789 kB/s | 27 kB 00:00 2023-11-28T11:48:51.546 INFO:teuthology.orchestra.run.smithi144.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-28T11:48:51.571 INFO:teuthology.orchestra.run.smithi144.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-11-28T11:48:51.663 INFO:teuthology.orchestra.run.smithi144.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-11-28T11:48:51.705 INFO:teuthology.orchestra.run.smithi144.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-11-28T11:48:51.789 INFO:teuthology.orchestra.run.smithi144.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-11-28T11:48:51.814 INFO:teuthology.orchestra.run.smithi144.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-11-28T11:48:51.839 INFO:teuthology.orchestra.run.smithi144.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-11-28T11:48:51.865 INFO:teuthology.orchestra.run.smithi144.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-28T11:48:51.890 INFO:teuthology.orchestra.run.smithi144.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 400 kB/s | 20 kB 00:00 2023-11-28T11:48:51.915 INFO:teuthology.orchestra.run.smithi144.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-28T11:48:51.949 INFO:teuthology.orchestra.run.smithi144.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-28T11:48:51.991 INFO:teuthology.orchestra.run.smithi144.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-28T11:48:52.133 INFO:teuthology.orchestra.run.smithi144.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-11-28T11:48:52.175 INFO:teuthology.orchestra.run.smithi144.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-28T11:48:52.217 INFO:teuthology.orchestra.run.smithi144.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-28T11:48:52.292 INFO:teuthology.orchestra.run.smithi144.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-11-28T11:48:52.384 INFO:teuthology.orchestra.run.smithi144.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-11-28T11:48:52.443 INFO:teuthology.orchestra.run.smithi144.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-11-28T11:48:52.501 INFO:teuthology.orchestra.run.smithi144.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-11-28T11:48:52.585 INFO:teuthology.orchestra.run.smithi144.stdout:(128/131): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-11-28T11:48:52.635 INFO:teuthology.orchestra.run.smithi144.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-28T11:48:52.960 INFO:teuthology.orchestra.run.smithi144.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 7.1 MB/s | 3.7 MB 00:00 2023-11-28T11:48:53.044 INFO:teuthology.orchestra.run.smithi144.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 6.6 MB/s | 3.6 MB 00:00 2023-11-28T11:48:53.046 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:48:53.046 INFO:teuthology.orchestra.run.smithi144.stdout:Total 14 MB/s | 203 MB 00:14 2023-11-28T11:48:53.888 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2023-11-28T11:48:54.007 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2023-11-28T11:48:54.007 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2023-11-28T11:48:54.327 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-11-28T11:48:54.493 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:48:54.651 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:48:54.844 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T11:48:55.016 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:48:55.118 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2023-11-28T11:48:55.122 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2023-11-28T11:48:55.153 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:48:55.367 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:48:55.773 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:48:56.266 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:48:56.664 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:48:56.866 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:48:57.002 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:48:57.158 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:48:57.257 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:48:57.450 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:48:57.686 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:48:57.886 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:48:58.271 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:48:58.528 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:48:58.733 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:48:58.838 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:48:59.016 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:48:59.099 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:48:59.215 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:48:59.358 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:48:59.490 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:49:00.063 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2023-11-28T11:49:00.108 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:49:00.274 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:49:00.329 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:49:00.522 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:49:00.690 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:49:00.826 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T11:49:00.849 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:49:01.023 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:49:01.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:49:01.423 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T11:49:01.664 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:49:01.733 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:49:01.995 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T11:49:02.043 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T11:49:02.233 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T11:49:02.248 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:49:02.384 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:49:02.435 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T11:49:02.581 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:49:02.612 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T11:49:02.665 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:49:02.813 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T11:49:02.836 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:49:03.016 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:49:03.018 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T11:49:03.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:49:03.284 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:49:03.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:49:03.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:49:03.865 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:49:04.102 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:49:04.236 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:49:04.298 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:49:04.407 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:49:04.547 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T11:49:04.580 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:49:04.726 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:49:04.776 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T11:49:04.910 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T11:49:05.035 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T11:49:05.078 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T11:49:05.247 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T11:49:05.265 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:49:05.380 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:49:05.563 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T11:49:05.930 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:49:06.548 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T11:49:06.730 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:49:07.035 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:49:07.320 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T11:49:07.374 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T11:49:07.530 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:49:07.592 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:49:07.774 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:49:07.975 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:49:08.118 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:49:08.215 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T11:49:08.274 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T11:49:08.464 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T11:49:08.475 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T11:49:08.641 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T11:49:08.644 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T11:49:08.810 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T11:49:08.864 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:49:08.988 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T11:49:09.028 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:49:09.157 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:49:09.185 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T11:49:09.427 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:49:09.541 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T11:49:09.638 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:49:09.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T11:49:10.007 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T11:49:10.242 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T11:49:10.451 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T11:49:10.754 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T11:49:10.997 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T11:49:11.198 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T11:49:11.392 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T11:49:11.420 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T11:49:11.587 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T11:49:11.753 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T11:49:12.063 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T11:49:12.395 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T11:49:12.408 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T11:49:12.719 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T11:49:12.794 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T11:49:12.989 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T11:49:13.028 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T11:49:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T11:49:13.234 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T11:49:13.367 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T11:49:13.449 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:49:13.563 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:49:13.624 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:49:13.641 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:49:13.820 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:49:13.865 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T11:49:14.002 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:49:14.067 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T11:49:14.144 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T11:49:14.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:49:14.277 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:49:14.297 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T11:49:14.469 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T11:49:14.470 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:49:14.549 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T11:49:14.623 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T11:49:14.772 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:49:14.775 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T11:49:14.952 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:49:14.990 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T11:49:15.154 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:49:15.236 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T11:49:15.279 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:49:15.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T11:49:15.495 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:49:15.642 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:49:15.649 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T11:49:15.808 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:49:15.817 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T11:49:16.006 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:49:16.014 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T11:49:16.193 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:49:16.198 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T11:49:16.383 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:49:16.532 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:49:16.641 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T11:49:16.677 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:49:16.837 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T11:49:16.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:49:17.077 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T11:49:17.108 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:49:17.293 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:49:17.455 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T11:49:17.543 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:49:17.713 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T11:49:17.795 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:49:17.914 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T11:49:18.021 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:49:18.124 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T11:49:18.246 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:49:18.350 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T11:49:18.687 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T11:49:18.903 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T11:49:19.080 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T11:49:19.293 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T11:49:19.355 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:49:19.451 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T11:49:19.616 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:49:19.837 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:49:19.907 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T11:49:20.129 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:49:20.191 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T11:49:20.416 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T11:49:20.420 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:49:20.601 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:49:20.638 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T11:49:20.811 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:49:20.973 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:49:21.204 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:49:21.398 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:49:21.418 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:49:21.536 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:49:21.548 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:49:21.613 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:49:21.721 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:49:21.812 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:49:21.894 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:49:22.106 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:49:22.288 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:49:22.439 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:49:22.584 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:49:22.800 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:49:23.007 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:49:23.192 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:49:23.433 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:49:23.652 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:49:23.802 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:49:23.999 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:49:25.098 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:49:25.335 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:49:25.339 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:49:25.536 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:49:25.560 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:49:25.834 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:49:26.128 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:49:26.130 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:49:26.359 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:49:26.419 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:49:26.514 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:49:26.661 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:49:27.153 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:49:27.318 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:49:27.378 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:49:27.552 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:49:31.202 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:49:31.418 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:49:31.997 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:49:32.706 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:49:32.791 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:49:32.926 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:49:45.343 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:49:45.343 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2023-11-28T11:49:45.343 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2023-11-28T11:49:45.343 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-11-28T11:49:45.343 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /run 2023-11-28T11:49:45.343 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2023-11-28T11:49:45.344 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2023-11-28T11:49:45.344 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2023-11-28T11:49:45.344 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2023-11-28T11:49:45.344 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-11-28T11:49:45.344 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:49:45.674 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:49:47.244 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:49:47.415 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:49:47.633 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:49:47.774 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:49:48.226 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:49:48.424 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:49:48.441 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:49:50.123 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:49:50.284 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:49:50.388 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:49:50.571 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:49:50.756 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:49:50.973 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:49:51.264 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /dev 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /run 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2023-11-28T11:49:51.944 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2023-11-28T11:49:51.945 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2023-11-28T11:49:51.945 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /dev 2023-11-28T11:49:51.945 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:49:52.227 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:49:52.281 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:49:52.406 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:49:52.495 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:49:52.858 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:49:53.031 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:49:53.066 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:49:53.239 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:49:53.251 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:49:53.847 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:49:54.002 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:49:54.221 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:49:54.364 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:49:54.516 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:49:54.726 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:49:54.760 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:49:54.816 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:49:54.906 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:49:54.947 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:49:55.031 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:49:55.048 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:49:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:49:55.457 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:49:55.622 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:49:56.736 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:49:56.913 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:49:57.028 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:49:57.201 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:49:57.388 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:49:57.604 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:49:57.918 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:49:58.881 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:49:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:49:59.109 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:49:59.454 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:49:59.643 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:49:59.677 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:49:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:49:59.866 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:50:00.040 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:50:00.335 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:50:00.488 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:50:00.678 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:50:01.117 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:50:01.359 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:50:01.396 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:50:01.554 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:50:01.570 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T11:50:01.596 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:50:01.771 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:50:01.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T11:50:02.093 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T11:50:02.117 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:50:02.263 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:50:02.301 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:50:02.491 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:50:03.006 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:50:04.818 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:50:05.071 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:50:05.089 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:50:05.300 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:50:05.554 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:50:05.816 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:50:06.097 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:50:06.262 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:50:06.271 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:50:06.447 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T11:50:06.612 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:50:06.698 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:50:06.699 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:50:06.895 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:50:06.908 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:50:06.908 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:50:07.073 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:50:07.073 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:50:07.291 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:50:07.469 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:50:07.553 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:50:07.877 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T11:50:07.970 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:50:07.970 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:50:07.970 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:50:07.970 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:50:07.970 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:50:07.970 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:50:07.971 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:50:07.971 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:50:07.971 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:50:07.971 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:50:07.971 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:50:07.971 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:50:07.972 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:50:07.973 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:50:07.973 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:50:07.973 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:50:07.973 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:50:07.973 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:50:07.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:50:07.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:50:07.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:50:07.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T11:50:07.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T11:50:07.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T11:50:07.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T11:50:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T11:50:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T11:50:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T11:50:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T11:50:07.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T11:50:07.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T11:50:07.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T11:50:07.993 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T11:50:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:50:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:50:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:50:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:50:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:50:07.994 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:50:08.121 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T11:50:08.368 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T11:50:08.528 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:50:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T11:50:08.965 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T11:50:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:50:09.262 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:50:10.589 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:50:10.590 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:45 ago on Tue 28 Nov 2023 11:48:25 AM UTC. 2023-11-28T11:50:11.459 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:50:11.711 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:50:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:50:11.878 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:50:12.161 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:50:12.441 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:50:12.824 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:50:13.002 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:50:13.086 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T11:50:13.151 INFO:teuthology.orchestra.run.smithi177.stdout:Installed products updated. 2023-11-28T11:50:13.279 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:50:13.290 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T11:50:13.381 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:13.381 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2023-11-28T11:50:13.381 INFO:teuthology.orchestra.run.smithi177.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:50:13.381 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.381 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.381 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.382 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.383 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:50:13.384 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:50:13.385 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:50:13.386 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:50:13.387 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:50:13.388 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:50:13.389 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:50:13.390 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:50:13.391 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:50:13.392 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:50:13.393 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:50:13.394 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:50:13.395 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:50:13.396 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:50:13.397 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:50:13.397 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:13.397 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-28T11:50:13.435 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:50:13.444 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T11:50:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T11:50:13.579 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:50:13.585 DEBUG:teuthology.parallel:result is None 2023-11-28T11:50:13.771 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:50:13.772 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:50:13.991 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:50:14.183 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:50:14.677 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:50:14.677 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:50:14.677 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:50:14.678 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:50:14.679 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:50:14.680 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:50:14.680 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:50:14.680 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:50:14.680 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:50:14.680 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:50:14.689 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:50:14.690 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:50:14.691 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T11:50:14.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T11:50:14.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T11:50:14.694 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T11:50:14.695 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T11:50:14.696 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:50:14.697 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:50:17.349 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:50:17.349 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:51 ago on Tue 28 Nov 2023 11:48:26 AM UTC. 2023-11-28T11:50:20.068 INFO:teuthology.orchestra.run.smithi144.stdout:Installed products updated. 2023-11-28T11:50:20.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:20.293 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2023-11-28T11:50:20.293 INFO:teuthology.orchestra.run.smithi144.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:50:20.293 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.294 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.295 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:50:20.296 INFO:teuthology.orchestra.run.smithi144.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.297 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:50:20.298 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:50:20.299 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:50:20.300 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:50:20.301 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:50:20.302 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:50:20.303 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:50:20.304 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:50:20.305 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:50:20.305 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:20.305 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2023-11-28T11:50:20.475 DEBUG:teuthology.parallel:result is None 2023-11-28T11:50:20.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:50:20.621 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:50:20.644 INFO:teuthology.orchestra.run.smithi144.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:50:20.645 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:50:20.645 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:50:20.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:50:20.784 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:50:20.813 INFO:teuthology.orchestra.run.smithi177.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:50:20.813 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:50:20.814 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:50:20.814 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T11:50:20.814 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:20.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:50:20.856 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:20.856 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:50:20.903 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T11:50:20.903 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:20.903 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:50:20.930 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:50:20.998 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:20.998 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:50:21.028 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:50:21.099 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T11:50:21.099 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:21.099 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:50:21.125 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:50:21.191 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:21.191 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:50:21.222 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:50:21.291 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T11:50:21.291 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:21.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:50:21.317 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:50:21.383 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:21.383 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:50:21.411 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:50:21.479 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T11:50:21.549 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T11:50:21.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2023-11-28T11:50:21.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2023-11-28T11:50:21.579 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T11:50:21.579 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:50:21.619 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:50:21.646 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T11:50:21.646 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:50:21.685 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:50:21.719 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T11:50:21.719 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T11:50:21.719 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T11:50:21.719 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:21.719 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:50:21.765 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:50:21.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:50:21.960 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:21.961 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:50:21.979 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:50:22.271 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:50:22.309 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T11:50:22.309 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T11:50:22.309 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469964', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': '17c45c858c4cb3105f973d12c6c74f8d5037aa89'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4140, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '17c45c858c4cb3105f973d12c6c74f8d5037aa89', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_11:27:19', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T11:50:22.310 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:50:22.312 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:50:22.362 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:50:22.370 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:50:22.432 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:22.432 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:50:22.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:50:22.454 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2023-11-28T11:50:22.512 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:50:22.512 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:22.512 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 390524 Links: 1 2023-11-28T11:50:22.513 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:22.513 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:22.513 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 11:50:14.381539713 +0000 2023-11-28T11:50:22.513 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 11:47:39.085962532 +0000 2023-11-28T11:50:22.513 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 11:47:39.085962532 +0000 2023-11-28T11:50:22.513 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T11:50:22.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:50:22.579 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T11:50:22.579 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T11:50:22.580 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000236683 s, 2.2 MB/s 2023-11-28T11:50:22.580 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:50:22.638 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2023-11-28T11:50:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:50:22.695 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:22.695 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 391284 Links: 1 2023-11-28T11:50:22.695 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:22.696 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:22.696 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 11:50:14.381539713 +0000 2023-11-28T11:50:22.696 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 11:47:39.602953782 +0000 2023-11-28T11:50:22.696 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 11:47:39.602953782 +0000 2023-11-28T11:50:22.696 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T11:50:22.696 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:50:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T11:50:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T11:50:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000230425 s, 2.2 MB/s 2023-11-28T11:50:22.762 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:50:22.820 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2023-11-28T11:50:22.878 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:50:22.878 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:22.879 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 385841 Links: 1 2023-11-28T11:50:22.879 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:22.879 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:22.879 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 11:50:14.382539696 +0000 2023-11-28T11:50:22.880 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 11:47:40.108945219 +0000 2023-11-28T11:50:22.880 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 11:47:40.108945219 +0000 2023-11-28T11:50:22.880 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T11:50:22.880 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:50:22.944 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T11:50:22.945 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T11:50:22.945 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000241092 s, 2.1 MB/s 2023-11-28T11:50:22.945 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:50:23.002 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 391528 Links: 1 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 11:50:14.382539696 +0000 2023-11-28T11:50:23.059 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 11:47:40.640936215 +0000 2023-11-28T11:50:23.060 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 11:47:40.640936215 +0000 2023-11-28T11:50:23.060 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T11:50:23.060 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:50:23.124 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T11:50:23.124 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T11:50:23.124 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000186759 s, 2.7 MB/s 2023-11-28T11:50:23.125 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:50:23.183 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:23.183 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:23.184 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:50:23.204 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:50:23.204 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2023-11-28T11:50:23.265 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:50:23.265 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:23.265 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 385017 Links: 1 2023-11-28T11:50:23.265 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:23.266 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:23.266 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 11:50:07.674867296 +0000 2023-11-28T11:50:23.266 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 11:47:22.759299544 +0000 2023-11-28T11:50:23.266 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 11:47:22.759299544 +0000 2023-11-28T11:50:23.266 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T11:50:23.267 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:50:23.333 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T11:50:23.334 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T11:50:23.334 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000264151 s, 1.9 MB/s 2023-11-28T11:50:23.335 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:50:23.395 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2023-11-28T11:50:23.455 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 390464 Links: 1 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 11:50:07.674867296 +0000 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 11:47:23.276285288 +0000 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 11:47:23.276285288 +0000 2023-11-28T11:50:23.456 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T11:50:23.456 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:50:23.524 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T11:50:23.524 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T11:50:23.524 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000238426 s, 2.1 MB/s 2023-11-28T11:50:23.525 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:50:23.585 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2023-11-28T11:50:23.643 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 394368 Links: 1 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 11:50:07.674867296 +0000 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 11:47:23.768271721 +0000 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 11:47:23.768271721 +0000 2023-11-28T11:50:23.644 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T11:50:23.644 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:50:23.710 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T11:50:23.710 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T11:50:23.710 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000215367 s, 2.4 MB/s 2023-11-28T11:50:23.711 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:50:23.769 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2023-11-28T11:50:23.826 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 390649 Links: 1 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-28 11:50:07.675867268 +0000 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-28 11:47:24.276257713 +0000 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-28 11:47:24.276257713 +0000 2023-11-28T11:50:23.827 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-28T11:50:23.827 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:50:23.893 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-28T11:50:23.893 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-28T11:50:23.893 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000247214 s, 2.1 MB/s 2023-11-28T11:50:23.894 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:50:23.952 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:23.953 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:50:23.953 INFO:tasks.ceph:Generating config... 2023-11-28T11:50:23.954 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T11:50:23.954 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T11:50:23.954 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T11:50:23.954 INFO:tasks.ceph:[client] pid file = 2023-11-28T11:50:23.954 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T11:50:23.955 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T11:50:23.955 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T11:50:23.955 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T11:50:23.956 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T11:50:23.956 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T11:50:23.956 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T11:50:23.956 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T11:50:23.956 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T11:50:23.956 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T11:50:23.956 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T11:50:23.956 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T11:50:23.956 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T11:50:23.956 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T11:50:23.956 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T11:50:23.956 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T11:50:23.957 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T11:50:23.958 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T11:50:23.958 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T11:50:24.018 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T11:50:24.022 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T11:50:24.070 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:50:24.098 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.144'), ('mon.b', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]')] 2023-11-28T11:50:24.098 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.144,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T11:50:24.099 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:24.099 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:50:24.156 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.144 --add b 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:24.245 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool: generated fsid af726c13-c3f1-4279-9fca-12c6e4621b18 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:setting min_mon_release = pacific 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 0 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:fsid af726c13-c3f1-4279-9fca-12c6e4621b18 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:last_changed 2023-11-28T11:50:24.245749+0000 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:created 2023-11-28T11:50:24.245749+0000 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:min_mon_release 16 (pacific) 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:election_strategy: 1 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.a 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2023-11-28T11:50:24.246 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T11:50:24.248 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:50:24.264 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID af726c13-c3f1-4279-9fca-12c6e4621b18... 2023-11-28T11:50:24.265 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:50:24.307 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2023-11-28T11:50:24.350 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.351 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2023-11-28T11:50:24.352 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.353 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.144,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on skipped message = False 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = af726c13-c3f1-4279-9fca-12c6e4621b18 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.354 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2023-11-28T11:50:24.355 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.356 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2023-11-28T11:50:24.357 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2023-11-28T11:50:24.358 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2023-11-28T11:50:24.358 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2023-11-28T11:50:24.358 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2023-11-28T11:50:24.358 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2023-11-28T11:50:24.358 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2023-11-28T11:50:24.359 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:50:24.360 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2023-11-28T11:50:24.361 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:50:24.362 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.363 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T11:50:24.364 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2023-11-28T11:50:24.365 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2023-11-28T11:50:24.366 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2023-11-28T11:50:24.368 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2023-11-28T11:50:24.368 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2023-11-28T11:50:24.368 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:50:24.368 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2023-11-28T11:50:24.368 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2023-11-28T11:50:24.369 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:50:24.370 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.371 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.144,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on skipped message = False 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = af726c13-c3f1-4279-9fca-12c6e4621b18 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.372 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.373 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2023-11-28T11:50:24.374 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:50:24.375 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:50:24.376 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:50:24.377 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.378 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:50:24.379 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-11-28T11:50:24.380 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds balancer = 20 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-11-28T11:50:24.381 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-11-28T11:50:24.382 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T11:50:24.382 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T11:50:24.465 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T11:50:24.465 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:24.465 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T11:50:24.482 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:24.482 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T11:50:24.538 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:50:24.538 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:24.538 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:50:24.538 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:50:24.614 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:24.614 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:24.670 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:50:24.671 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:24.671 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:50:24.671 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:50:24.709 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:24.709 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:24.766 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T11:50:24.766 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T11:50:24.766 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:50:24.819 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:50:24.821 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:50:24.882 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:50:24.885 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T11:50:24.886 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:50:24.940 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:50:24.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T11:50:24.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:50:25.061 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:50:25.064 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T11:50:25.090 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:50:25.145 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:50:25.148 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T11:50:25.174 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:50:25.267 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:50:25.269 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T11:50:25.296 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T11:50:25.296 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T11:50:25.341 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T11:50:25.354 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T11:50:25.437 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T11:50:25.450 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T11:50:25.493 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T11:50:25.505 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T11:50:25.547 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T11:50:25.560 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T11:50:25.560 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:50:25.560 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T11:50:25.587 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:25.587 INFO:tasks.ceph:role: osd.0 2023-11-28T11:50:25.587 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:50:25.587 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:25.662 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:25.663 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:25.663 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:26.096 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T11:50:26.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T11:50:26.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T11:50:26.289 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T11:50:26.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T11:50:26.389 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:26.389 INFO:tasks.ceph:role: osd.1 2023-11-28T11:50:26.389 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:50:26.389 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:50:26.463 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:26.463 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:26.463 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:26.463 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:26.463 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:26.464 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:26.464 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:26.464 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:26.464 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:26.880 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T11:50:26.882 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T11:50:26.883 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T11:50:26.991 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T11:50:27.022 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T11:50:27.090 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:27.090 INFO:tasks.ceph:role: osd.2 2023-11-28T11:50:27.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:50:27.090 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:50:27.163 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:27.163 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:27.164 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:27.164 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:27.164 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:27.164 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:27.164 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:27.612 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T11:50:27.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T11:50:27.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T11:50:27.749 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T11:50:27.780 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T11:50:27.848 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:27.848 INFO:tasks.ceph:role: osd.3 2023-11-28T11:50:27.848 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:50:27.849 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:50:27.923 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:27.923 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:27.923 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:27.924 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:28.371 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T11:50:28.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T11:50:28.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T11:50:28.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T11:50:28.601 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:28.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:28.879+0000 7eff2bbb6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T11:50:28.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:28.879+0000 7eff2bbb6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T11:50:28.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:28.879+0000 7eff2bbb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:50:28.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:28.879+0000 7eff2bbb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:50:28.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:28.879+0000 7eff2bbb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:50:28.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:28.879+0000 7eff2bbb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T11:50:31.755 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T11:50:31.785 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:31.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:31.874+0000 7fc7fe2ba640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T11:50:31.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:31.874+0000 7fc7fe2ba640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T11:50:31.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:31.874+0000 7fc7fe2ba640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:50:31.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:31.874+0000 7fc7fe2ba640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:50:31.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:31.874+0000 7fc7fe2ba640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:50:31.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:31.875+0000 7fc7fe2ba640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T11:50:34.750 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T11:50:34.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:34.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:34.946+0000 7f855132a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T11:50:34.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:34.947+0000 7f855132a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T11:50:34.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:34.947+0000 7f855132a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:50:34.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:34.947+0000 7f855132a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:50:34.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:34.947+0000 7f855132a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:50:34.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:34.947+0000 7f855132a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T11:50:37.820 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T11:50:37.852 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:37.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:37.942+0000 7f3ef5e05640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T11:50:37.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:37.942+0000 7f3ef5e05640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T11:50:37.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:37.942+0000 7f3ef5e05640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:50:37.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:37.942+0000 7f3ef5e05640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:50:37.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:37.942+0000 7f3ef5e05640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:50:37.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:37.943+0000 7f3ef5e05640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T11:50:40.816 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T11:50:40.848 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T11:50:40.881 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:40.881 INFO:tasks.ceph:role: osd.4 2023-11-28T11:50:40.881 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:50:40.881 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:50:40.959 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:40.959 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:40.959 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:40.959 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:40.960 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:40.960 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:40.960 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:40.960 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:40.960 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:40.960 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:41.390 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T11:50:41.392 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T11:50:41.392 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T11:50:41.564 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T11:50:41.603 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T11:50:41.674 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:41.674 INFO:tasks.ceph:role: osd.5 2023-11-28T11:50:41.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:50:41.675 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:50:41.750 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:41.750 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:41.750 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:41.750 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:41.750 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:41.751 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:41.751 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:41.751 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:41.751 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:41.751 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:42.172 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T11:50:42.175 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T11:50:42.175 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T11:50:42.284 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T11:50:42.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T11:50:42.390 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:42.390 INFO:tasks.ceph:role: osd.6 2023-11-28T11:50:42.390 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:50:42.390 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:50:42.465 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:42.466 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:42.907 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T11:50:42.908 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T11:50:42.909 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T11:50:43.059 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T11:50:43.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T11:50:43.164 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:50:43.164 INFO:tasks.ceph:role: osd.7 2023-11-28T11:50:43.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:50:43.164 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:50:43.240 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:50:43.241 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:50:43.682 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-11-28T11:50:43.684 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-11-28T11:50:43.684 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T11:50:43.869 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T11:50:43.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:44.020 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:44.019+0000 7f91e704c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T11:50:44.021 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:44.019+0000 7f91e704c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T11:50:44.021 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:44.019+0000 7f91e704c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:50:44.021 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:44.019+0000 7f91e704c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:50:44.021 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:44.019+0000 7f91e704c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:50:44.021 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:44.019+0000 7f91e704c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T11:50:46.882 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T11:50:46.914 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:47.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:47.004+0000 7f32f37d3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T11:50:47.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:47.005+0000 7f32f37d3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T11:50:47.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:47.005+0000 7f32f37d3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:50:47.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:47.005+0000 7f32f37d3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:50:47.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:47.005+0000 7f32f37d3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:50:47.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:47.005+0000 7f32f37d3640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T11:50:49.880 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T11:50:49.910 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:50.000 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:49.998+0000 7f75d5437640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T11:50:50.000 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:49.999+0000 7f75d5437640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T11:50:50.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:49.999+0000 7f75d5437640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:50:50.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:49.999+0000 7f75d5437640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:50:50.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:49.999+0000 7f75d5437640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:50:50.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:49.999+0000 7f75d5437640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T11:50:52.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:50:52.314 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:50:52.874 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T11:50:52.905 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:52.996 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:52.994+0000 7f46ec970640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T11:50:52.996 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:52.994+0000 7f46ec970640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T11:50:52.996 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:52.994+0000 7f46ec970640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:50:52.996 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:52.994+0000 7f46ec970640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:50:52.996 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:52.994+0000 7f46ec970640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:50:52.996 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:52.995+0000 7f46ec970640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T11:50:55.857 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T11:50:55.889 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T11:50:55.889 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:55.889 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T11:50:55.921 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:55.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T11:50:55.991 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:55.991 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T11:50:56.061 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:56.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T11:50:56.130 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:56.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T11:50:56.199 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:56.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T11:50:56.267 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:56.267 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T11:50:56.334 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.334 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T11:50:56.366 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.366 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T11:50:56.436 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.436 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T11:50:56.504 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.504 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T11:50:56.572 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.572 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T11:50:56.639 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T11:50:56.707 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.707 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T11:50:56.773 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:56.773 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T11:50:56.792 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:56.792 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T11:50:56.850 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.850 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T11:50:56.867 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:56.867 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T11:50:56.924 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T11:50:56.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:50:56.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:50:56.967 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2023-11-28T11:50:56.967 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAA1GVlU3zUMBAAYSJLQSW0CAYPWfcdqxqHbA== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAA1GVlzuwFOBAAtN3Mhkt5bwjiPFumLWf5LQ== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVlyo2xAxAARq98KYTF0ViGGgRsgbiHJg== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAE1GVlRYZuNBAAkRLYVWzVGa/wVN7KApc9pA== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAH1GVlzE4mNBAA8fJmDI1X35Kb7ffo7fKFGQ== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAK1GVl0BB5OBAAJXjaD/H42vlurqW5ReH8QQ== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAN1GVllXIyOBAAHYoY70u/C1a0B/IZhBZ3kw== 2023-11-28T11:50:56.968 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAA1GVl+4ibNBAAmnsKW1HqPNEAHCtefjHz4A== 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVlmkasCBAAaYU+0cOXvyDnawSPLqNQBg== 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVltR3sDxAAkRdNJ3ar7Vh/OwKimeVIGQ== 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAU1GVlWMw2ARAAQDzOPIY+tj6nPKoQ5d1Opw== 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAX1GVlwfFcABAAzJkyTNckY6Ua/4KuHrTn2A== 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAa1GVldpcCABAAY7IyYyuR3TMLzueAnd70vA== 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2023-11-28T11:50:56.969 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAc1GVlnehWOxAA+UrMHeuJP4Btd7AlMXMyYA== 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVl4NRbFBAA6tNlYbw6YiCQ4X7ewAOxAw== 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout:[client.2] 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVlxfcVGhAAEX5yipIE6u3NvW4h8fLhjw== 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVlfaJnHRAA4AdUVgWb3JW1D50Vj+quDQ== 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout:[client.3] 2023-11-28T11:50:56.970 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB1GVlF0KoIBAAewnQNQ6UxhSPTRCTU/mgtQ== 2023-11-28T11:50:56.989 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2023-11-28T11:50:56.989 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAA1GVlU3zUMBAAYSJLQSW0CAYPWfcdqxqHbA== 2023-11-28T11:50:56.989 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAA1GVlzuwFOBAAtN3Mhkt5bwjiPFumLWf5LQ== 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVlyo2xAxAARq98KYTF0ViGGgRsgbiHJg== 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAE1GVlRYZuNBAAkRLYVWzVGa/wVN7KApc9pA== 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAH1GVlzE4mNBAA8fJmDI1X35Kb7ffo7fKFGQ== 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAK1GVl0BB5OBAAJXjaD/H42vlurqW5ReH8QQ== 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAN1GVllXIyOBAAHYoY70u/C1a0B/IZhBZ3kw== 2023-11-28T11:50:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAA1GVl+4ibNBAAmnsKW1HqPNEAHCtefjHz4A== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVlmkasCBAAaYU+0cOXvyDnawSPLqNQBg== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVltR3sDxAAkRdNJ3ar7Vh/OwKimeVIGQ== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAU1GVlWMw2ARAAQDzOPIY+tj6nPKoQ5d1Opw== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAX1GVlwfFcABAAzJkyTNckY6Ua/4KuHrTn2A== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAa1GVldpcCABAAY7IyYyuR3TMLzueAnd70vA== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAc1GVlnehWOxAA+UrMHeuJP4Btd7AlMXMyYA== 2023-11-28T11:50:56.991 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVl4NRbFBAA6tNlYbw6YiCQ4X7ewAOxAw== 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout:[client.2] 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVlxfcVGhAAEX5yipIE6u3NvW4h8fLhjw== 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVlfaJnHRAA4AdUVgWb3JW1D50Vj+quDQ== 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout:[client.3] 2023-11-28T11:50:56.992 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQAB1GVlF0KoIBAAewnQNQ6UxhSPTRCTU/mgtQ== 2023-11-28T11:50:56.993 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:50:57.014 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:50:57.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.088 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.146 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.204 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.256 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.258 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.313 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.371 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:50:57.426 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:50:57.479 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.481 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.538 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:50:57.594 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.596 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.649 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.704 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.706 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.759 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.761 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:50:57.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.816 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.872 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.874 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.930 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.933 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:57.991 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:50:58.047 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T11:50:58.048 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T11:50:58.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:50:58.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T11:50:58.200 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T11:50:58.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:50:58.325 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T11:50:58.354 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T11:50:58.421 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:50:58.519 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T11:50:58.546 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:58.548 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:50:58.603 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T11:50:58.603 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T11:50:58.603 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T11:50:58.605 INFO:tasks.ceph.mon.a:Started 2023-11-28T11:50:58.605 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T11:50:58.606 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T11:50:58.645 INFO:tasks.ceph.mon.b:Started 2023-11-28T11:50:58.645 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T11:50:58.646 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T11:50:58.647 INFO:tasks.ceph.mon.c:Started 2023-11-28T11:50:58.647 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T11:50:58.648 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T11:50:58.648 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T11:50:58.650 INFO:tasks.ceph.mgr.y:Started 2023-11-28T11:50:58.650 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T11:50:58.650 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T11:50:58.652 INFO:tasks.ceph.mgr.x:Started 2023-11-28T11:50:58.652 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T11:50:58.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T11:50:58.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.772+0000 7f7575f54700 1 Processor -- start 2023-11-28T11:50:58.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.772+0000 7f7575f54700 1 -- start start 2023-11-28T11:50:58.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f7575f54700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 0x7f7570140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f7575f54700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757013ff00 con 0x7f757006f850 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f7575f54700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75701404c0 con 0x7f75700cdc90 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f7575f54700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757015aa80 con 0x7f75700cd8c0 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 0x7f7570140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 0x7f7570140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33712/0 (socket says 172.21.15.144:33712) 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756ffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f757006f850 0x7f757006fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50428/0 (socket says 172.21.15.144:50428) 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 -- 172.21.15.144:0/2787763808 learned_addr learned my addr 172.21.15.144:0/2787763808 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 -- 172.21.15.144:0/2787763808 >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 msgr2=0x7f7570140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 -- 172.21.15.144:0/2787763808 >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 msgr2=0x7f7570140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 --2- 172.21.15.144:0/2787763808 >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 0x7f7570140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:50:58.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.773+0000 7f756f7fe700 1 --2- 172.21.15.144:0/2787763808 >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 0x7f7570140950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:50:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579845708 0 0) 0x7f757013ff00 con 0x7f757006f850 2023-11-28T11:50:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75580035b0 con 0x7f757006f850 2023-11-28T11:50:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385863209 0 0) 0x7f75701404c0 con 0x7f75700cdc90 2023-11-28T11:50:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f757013ff00 con 0x7f75700cdc90 2023-11-28T11:50:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189763931 0 0) 0x7f757013ff00 con 0x7f75700cdc90 2023-11-28T11:50:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 msgr2=0x7f7570140950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 --2- 172.21.15.144:0/2787763808 >> v2:172.21.15.144:3300/0 conn(0x7f75700cd8c0 0x7f7570140950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 >> v1:172.21.15.144:6789/0 conn(0x7f757006f850 legacy=0x7f757006fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700b3040 con 0x7f75700cdc90 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.962+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f7560001670 con 0x7f75700cdc90 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 >> v1:172.21.15.177:6789/0 conn(0x7f75700cdc90 legacy=0x7f75700b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558003b90 0x7f7558003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7558004850 0x7f7558004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7558005270 0x7f755800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757015aa80 con 0x7f7558003b90 2023-11-28T11:50:58.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755800c1b0 con 0x7f7558005270 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755800c370 con 0x7f7558004850 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7558005270 0x7f755800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756f7fe700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558003b90 0x7f7558003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756effd700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7558004850 0x7f7558004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 -- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7558004850 msgr2=0x7f7558004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7558004850 0x7f7558004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:58.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 -- 172.21.15.144:0/2787763808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558003b90 msgr2=0x7f7558003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558003b90 0x7f7558003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 -- 172.21.15.144:0/2787763808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757013ff00 con 0x7f7558005270 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756f7fe700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558003b90 0x7f7558003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 -- 172.21.15.144:0/2787763808 reap_dead start 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.963+0000 7f756ffff700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7558005270 0x7f755800ba10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f756800c2f0 tx=0x7f7558005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4528cfbf601c17bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.964+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568001f30 con 0x7f7558005270 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.964+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800ce90 con 0x7f7558005270 2023-11-28T11:50:58.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.964+0000 7f756e7fc700 1 -- 172.21.15.144:0/2787763808 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680072d0 con 0x7f7558005270 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 -- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7558005270 msgr2=0x7f755800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7558005270 0x7f755800ba10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f756800c2f0 tx=0x7f7558005880 comp rx=0 tx=0).stop 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 -- 172.21.15.144:0/2787763808 shutdown_connections 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 --2- 172.21.15.144:0/2787763808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7558005270 0x7f755800ba10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 -- 172.21.15.144:0/2787763808 >> 172.21.15.144:0/2787763808 conn(0x7f757006aed0 msgr2=0x7f757006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 -- 172.21.15.144:0/2787763808 shutdown_connections 2023-11-28T11:50:58.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.965+0000 7f7575f54700 1 -- 172.21.15.144:0/2787763808 wait complete. 2023-11-28T11:50:58.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.967+0000 7f7575f54700 1 Processor -- start 2023-11-28T11:50:58.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.967+0000 7f7575f54700 1 -- start start 2023-11-28T11:50:58.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f7575f54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 0x7f7570155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f7575f54700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75700b58e0 0x7f75700b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f7575f54700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75700b1720 0x7f75701f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f7575f54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75700b1bc0 con 0x7f7558005270 2023-11-28T11:50:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f7575f54700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75700b3d00 con 0x7f75700b58e0 2023-11-28T11:50:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f7575f54700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700b3f50 con 0x7f75700b1720 2023-11-28T11:50:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f756f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 0x7f7570155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f756f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 0x7f7570155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33744/0 (socket says 172.21.15.144:33744) 2023-11-28T11:50:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f756f7fe700 1 -- 172.21.15.144:0/649936305 learned_addr learned my addr 172.21.15.144:0/649936305 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f756effd700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75700b58e0 0x7f75700b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.968+0000 7f756ffff700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75700b1720 0x7f75701f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756f7fe700 1 -- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75700b1720 msgr2=0x7f75701f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756f7fe700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75700b1720 0x7f75701f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756f7fe700 1 -- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75700b58e0 msgr2=0x7f75700b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756f7fe700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75700b58e0 0x7f75700b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756f7fe700 1 -- 172.21.15.144:0/649936305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700b35f0 con 0x7f7558005270 2023-11-28T11:50:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756effd700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75700b58e0 0x7f75700b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:50:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f756f7fe700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 0x7f7570155010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f756400f4e0 tx=0x7f756400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:50:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756400c070 con 0x7f7558005270 2023-11-28T11:50:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564004020 con 0x7f7558005270 2023-11-28T11:50:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564004390 con 0x7f7558005270 2023-11-28T11:50:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75700b3310 con 0x7f7558005270 2023-11-28T11:50:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7570140150 con 0x7f7558005270 2023-11-28T11:50:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f75640036a0 con 0x7f7558005270 2023-11-28T11:50:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.969+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f75640148b0 con 0x7f7558005270 2023-11-28T11:50:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.970+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7570068690 con 0x7f7558005270 2023-11-28T11:50:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:58.974+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f7564018030 con 0x7f7558005270 2023-11-28T11:50:59.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.048+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7570140a80 con 0x7f7558005270 2023-11-28T11:50:59.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.051+0000 7f7557fff700 1 -- 172.21.15.144:0/649936305 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7564003b10 con 0x7f7558005270 2023-11-28T11:50:59.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 msgr2=0x7f7570155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 0x7f7570155010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f756400f4e0 tx=0x7f756400f7f0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 shutdown_connections 2023-11-28T11:50:59.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7558005270 0x7f7570155010 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75700b1720 0x7f75701f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 --2- 172.21.15.144:0/649936305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75700b58e0 0x7f75700b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 >> 172.21.15.144:0/649936305 conn(0x7f757006aed0 msgr2=0x7f757013ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:50:59.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 shutdown_connections 2023-11-28T11:50:59.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:50:59.052+0000 7f7575f54700 1 -- 172.21.15.144:0/649936305 wait complete. 2023-11-28T11:50:59.056 INFO:teuthology.orchestra.run.smithi144.stderr:adjusted tunables profile to default 2023-11-28T11:50:59.070 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T11:50:59.070 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T11:50:59.071 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:59.071 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T11:50:59.081 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:50:59.079+0000 7f6e7a3cf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:50:59.087 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:50:59.085+0000 7fa453583200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:50:59.098 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:59.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T11:50:59.145 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:50:59.143+0000 7f6e7a3cf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:50:59.155 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:50:59.154+0000 7fa453583200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:50:59.164 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:59.164 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T11:50:59.219 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:50:59.218+0000 7f6e7a3cf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:50:59.230 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:50:59.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T11:50:59.235 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:50:59.233+0000 7fa453583200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:50:59.261 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:59.261 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T11:50:59.287 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:59.287 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T11:50:59.299 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:50:59.298+0000 7f6e7a3cf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:50:59.318 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:50:59.317+0000 7fa453583200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:50:59.324 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:59.325 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T11:50:59.390 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:50:59.391 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T11:50:59.447 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:50:59.446+0000 7f6e7a3cf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:50:59.456 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 04af8260-efb0-4064-89be-509ad55c21c3 0 2023-11-28T11:50:59.461 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:50:59.459+0000 7fa453583200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:50:59.617 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.615+0000 7f25838e0700 1 Processor -- start 2023-11-28T11:50:59.617 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.616+0000 7f25838e0700 1 -- start start 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f25838e0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f257c06dee0 0x7f257c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f25838e0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f257c160540 0x7f257c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f25838e0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257c15d060 con 0x7f257c071040 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f25838e0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c15d340 con 0x7f257c06dee0 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f25838e0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c15d620 con 0x7f257c160540 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f257c160540 0x7f257c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f257c160540 0x7f257c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40234/0 (socket says 172.21.15.177:40234) 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 -- 172.21.15.177:0/2709135369 learned_addr learned my addr 172.21.15.177:0/2709135369 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2580e7b700 1 --2- 172.21.15.177:0/2709135369 >> v2:172.21.15.144:3300/0 conn(0x7f257c06dee0 0x7f257c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.620 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 -- 172.21.15.177:0/2709135369 >> v2:172.21.15.144:3300/0 conn(0x7f257c06dee0 msgr2=0x7f257c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 --2- 172.21.15.177:0/2709135369 >> v2:172.21.15.144:3300/0 conn(0x7f257c06dee0 0x7f257c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2573fff700 1 -- 172.21.15.177:0/2709135369 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74292236 0 0) 0x7f257c15d060 con 0x7f257c071040 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 -- 172.21.15.177:0/2709135369 >> v1:172.21.15.144:6789/0 conn(0x7f257c071040 legacy=0x7f257c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 -- 172.21.15.177:0/2709135369 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c15d900 con 0x7f257c160540 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.617+0000 7f2581e7d700 1 --2- 172.21.15.177:0/2709135369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c160540 0x7f257c160980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f257c0c3a10 tx=0x7f257400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11712335efb92327 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f2573fff700 1 -- 172.21.15.177:0/2709135369 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400e070 con 0x7f257c160540 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f2573fff700 1 -- 172.21.15.177:0/2709135369 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257400b910 con 0x7f257c160540 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f2573fff700 1 -- 172.21.15.177:0/2709135369 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257400bcc0 con 0x7f257c160540 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 -- 172.21.15.177:0/2709135369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c160540 msgr2=0x7f257c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 --2- 172.21.15.177:0/2709135369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c160540 0x7f257c160980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f257c0c3a10 tx=0x7f257400b330 comp rx=0 tx=0).stop 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 -- 172.21.15.177:0/2709135369 shutdown_connections 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 --2- 172.21.15.177:0/2709135369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c160540 0x7f257c160980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 --2- 172.21.15.177:0/2709135369 >> v2:172.21.15.144:3300/0 conn(0x7f257c06dee0 0x7f257c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.621 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 -- 172.21.15.177:0/2709135369 >> 172.21.15.177:0/2709135369 conn(0x7f257c0d0000 msgr2=0x7f257c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:50:59.622 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 -- 172.21.15.177:0/2709135369 shutdown_connections 2023-11-28T11:50:59.622 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.618+0000 7f25838e0700 1 -- 172.21.15.177:0/2709135369 wait complete. 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.620+0000 7f25838e0700 1 Processor -- start 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.620+0000 7f25838e0700 1 -- start start 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f25838e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f257c06dee0 0x7f257c1486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f25838e0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 0x7f257c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f25838e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f257c160540 0x7f257c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f25838e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c148c10 con 0x7f257c06dee0 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f25838e0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b3170 con 0x7f257c071040 2023-11-28T11:50:59.624 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f25838e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b3450 con 0x7f257c160540 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2580e7b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 0x7f257c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2580e7b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 0x7f257c15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40238/0 (socket says 172.21.15.177:40238) 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2580e7b700 1 -- 172.21.15.177:0/110695670 learned_addr learned my addr 172.21.15.177:0/110695670 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f257c160540 0x7f257c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f258167c700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f257c06dee0 0x7f257c1486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 -- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 msgr2=0x7f257c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 0x7f257c15b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 -- 172.21.15.177:0/110695670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f257c06dee0 msgr2=0x7f257c1486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f257c06dee0 0x7f257c1486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 -- 172.21.15.177:0/110695670 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009b30 con 0x7f257c160540 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2580e7b700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 0x7f257c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2581e7d700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f257c160540 0x7f257c0c50b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2574015000 tx=0x7f2574004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.621+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740092e0 con 0x7f257c160540 2023-11-28T11:50:59.625 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.622+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574004040 con 0x7f257c160540 2023-11-28T11:50:59.626 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.622+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574004410 con 0x7f257c160540 2023-11-28T11:50:59.626 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.622+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c15ccd0 con 0x7f257c160540 2023-11-28T11:50:59.626 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.622+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c06e870 con 0x7f257c160540 2023-11-28T11:50:59.626 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.622+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2574004570 con 0x7f257c160540 2023-11-28T11:50:59.626 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.622+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2574024410 con 0x7f257c160540 2023-11-28T11:50:59.626 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.623+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c06a270 con 0x7f257c160540 2023-11-28T11:50:59.628 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.627+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2574042b00 con 0x7f257c160540 2023-11-28T11:50:59.711 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.709+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "04af8260-efb0-4064-89be-509ad55c21c3", "id": 0} v 0) v1 -- 0x7f257c15d060 con 0x7f257c160540 2023-11-28T11:50:59.714 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.713+0000 7f2571ffb700 1 -- 172.21.15.177:0/110695670 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "04af8260-efb0-4064-89be-509ad55c21c3", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2574031070 con 0x7f257c160540 2023-11-28T11:50:59.714 INFO:teuthology.orchestra.run.smithi177.stdout:0 2023-11-28T11:50:59.715 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f257c160540 msgr2=0x7f257c0c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.715 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f257c160540 0x7f257c0c50b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2574015000 tx=0x7f2574004ad0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.715 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 shutdown_connections 2023-11-28T11:50:59.716 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f257c06dee0 0x7f257c1486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.716 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f257c160540 0x7f257c0c50b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.716 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 --2- 172.21.15.177:0/110695670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f257c071040 0x7f257c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.716 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 >> 172.21.15.177:0/110695670 conn(0x7f257c0d0000 msgr2=0x7f257c15e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:50:59.716 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 shutdown_connections 2023-11-28T11:50:59.716 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.714+0000 7f25838e0700 1 -- 172.21.15.177:0/110695670 wait complete. 2023-11-28T11:50:59.727 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 8fdb2107-bea4-4a2c-b202-a51d01193e51 1 2023-11-28T11:50:59.848 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.846+0000 7ff477f70700 1 Processor -- start 2023-11-28T11:50:59.848 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.847+0000 7ff477f70700 1 -- start start 2023-11-28T11:50:59.851 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff477f70700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff470154e70 0x7ff4700cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.851 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff477f70700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff47015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.851 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff477f70700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff470155240 con 0x7ff4700cf360 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff477f70700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47015b100 con 0x7ff470154e70 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff477f70700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015b3e0 con 0x7ff470071750 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff470154e70 0x7ff4700cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff47650d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff47015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff470154e70 0x7ff4700cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40242/0 (socket says 172.21.15.177:40242) 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 -- 172.21.15.177:0/3913858840 learned_addr learned my addr 172.21.15.177:0/3913858840 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 -- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 msgr2=0x7ff47015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 --2- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff47015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 -- 172.21.15.177:0/3913858840 >> v1:172.21.15.144:6789/0 conn(0x7ff4700cf360 legacy=0x7ff470071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff475d0c700 1 -- 172.21.15.177:0/3913858840 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47015b6c0 con 0x7ff470154e70 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.848+0000 7ff47650d700 1 --2- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff47015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:50:59.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.849+0000 7ff475d0c700 1 --2- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff470154e70 0x7ff4700cdbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0098f0 tx=0x7ff46c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86f29441242739e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.849+0000 7ff474d0a700 1 -- 172.21.15.177:0/3913858840 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011830 con 0x7ff470154e70 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.849+0000 7ff474d0a700 1 -- 172.21.15.177:0/3913858840 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c011990 con 0x7ff470154e70 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.849+0000 7ff474d0a700 1 -- 172.21.15.177:0/3913858840 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011d20 con 0x7ff470154e70 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 -- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff470154e70 msgr2=0x7ff4700cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 --2- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff470154e70 0x7ff4700cdbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0098f0 tx=0x7ff46c009c00 comp rx=0 tx=0).stop 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 -- 172.21.15.177:0/3913858840 shutdown_connections 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 --2- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff47015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 --2- 172.21.15.177:0/3913858840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff470154e70 0x7ff4700cdbd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 -- 172.21.15.177:0/3913858840 >> 172.21.15.177:0/3913858840 conn(0x7ff4700d0000 msgr2=0x7ff4700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 -- 172.21.15.177:0/3913858840 shutdown_connections 2023-11-28T11:50:59.853 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.850+0000 7ff477f70700 1 -- 172.21.15.177:0/3913858840 wait complete. 2023-11-28T11:50:59.854 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.852+0000 7ff477f70700 1 Processor -- start 2023-11-28T11:50:59.854 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.852+0000 7ff477f70700 1 -- start start 2023-11-28T11:50:59.856 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff477f70700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff4701f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.856 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff477f70700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 0x7ff4700cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.856 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff477f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff470154e70 0x7ff4700ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff477f70700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4701f1360 con 0x7ff470154e70 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff477f70700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff470156640 con 0x7ff4700cf360 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff477f70700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff470155d50 con 0x7ff470071750 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 0x7ff4700cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff475d0c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff4701f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 0x7ff4700cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40250/0 (socket says 172.21.15.177:40250) 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 -- 172.21.15.177:0/575580979 learned_addr learned my addr 172.21.15.177:0/575580979 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47650d700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff470154e70 0x7ff4700ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 -- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 msgr2=0x7ff4701f0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff4701f0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 -- 172.21.15.177:0/575580979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff470154e70 msgr2=0x7ff4700ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff470154e70 0x7ff4700ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.857 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.853+0000 7ff47550b700 1 -- 172.21.15.177:0/575580979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff468009a70 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff47550b700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 0x7ff4700cc570 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c4e0 tx=0x7ff46000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600052d0 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460005430 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c009550 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600038c0 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4700af520 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff460019030 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.854+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff460011850 con 0x7ff4700cf360 2023-11-28T11:50:59.858 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.855+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47015b9a0 con 0x7ff4700cf360 2023-11-28T11:50:59.859 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.857+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff460017070 con 0x7ff4700cf360 2023-11-28T11:50:59.941 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.940+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8fdb2107-bea4-4a2c-b202-a51d01193e51", "id": 1} v 0) v1 -- 0x7ff47015adb0 con 0x7ff4700cf360 2023-11-28T11:50:59.950 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.949+0000 7ff4667fc700 1 -- 172.21.15.177:0/575580979 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fdb2107-bea4-4a2c-b202-a51d01193e51", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff460003a20 con 0x7ff4700cf360 2023-11-28T11:50:59.951 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.949+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 msgr2=0x7ff4700cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.949+0000 7ff477f70700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 0x7ff4700cc570 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c4e0 tx=0x7ff46000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.949+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 shutdown_connections 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.949+0000 7ff477f70700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff470154e70 0x7ff4700ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.950+0000 7ff477f70700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff470071750 0x7ff4701f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.950+0000 7ff477f70700 1 --2- 172.21.15.177:0/575580979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4700cf360 0x7ff4700cc570 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.950+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 >> 172.21.15.177:0/575580979 conn(0x7ff4700d0000 msgr2=0x7ff47015ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:50:59.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.950+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 shutdown_connections 2023-11-28T11:50:59.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:50:59.950+0000 7ff477f70700 1 -- 172.21.15.177:0/575580979 wait complete. 2023-11-28T11:50:59.963 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 9ca99a48-48b5-41bc-bade-5d9fdb9bef65 2 2023-11-28T11:51:00.086 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.084+0000 7f0659d58700 1 Processor -- start 2023-11-28T11:51:00.086 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.084+0000 7f0659d58700 1 -- start start 2023-11-28T11:51:00.088 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.085+0000 7f0659d58700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 0x7f06540af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.088 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.085+0000 7f0659d58700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06540cc5f0 con 0x7f065406ba90 2023-11-28T11:51:00.088 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.085+0000 7f0659d58700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065415d470 con 0x7f06540cbdf0 2023-11-28T11:51:00.088 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.085+0000 7f0659d58700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065415d750 con 0x7f065415cef0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06537fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 0x7f06540af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f0652ffd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f06540cbdf0 0x7f065406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:44604/0 (socket says 172.21.15.177:44604) 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f0652ffd700 1 -- 172.21.15.177:0/1480239099 learned_addr learned my addr 172.21.15.177:0/1480239099 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06537fe700 1 --2- 172.21.15.177:0/1480239099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 0x7f06540af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48712/0 (socket says 172.21.15.177:48712) 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06537fe700 1 -- 172.21.15.177:0/1480239099 >> v1:172.21.15.177:6789/0 conn(0x7f06540cbdf0 legacy=0x7f065406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06527fc700 1 -- 172.21.15.177:0/1480239099 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816254956 0 0) 0x7f065415d470 con 0x7f06540cbdf0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06537fe700 1 -- 172.21.15.177:0/1480239099 >> v1:172.21.15.144:6789/0 conn(0x7f065406ba90 legacy=0x7f06540ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06537fe700 1 -- 172.21.15.177:0/1480239099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065415da30 con 0x7f065415cef0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06537fe700 1 --2- 172.21.15.177:0/1480239099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 0x7f06540af400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0644009dc0 tx=0x7f064400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fec3c445d213e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06527fc700 1 -- 172.21.15.177:0/1480239099 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064400bdd0 con 0x7f065415cef0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06527fc700 1 -- 172.21.15.177:0/1480239099 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0644011460 con 0x7f065415cef0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.086+0000 7f06527fc700 1 -- 172.21.15.177:0/1480239099 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644011810 con 0x7f065415cef0 2023-11-28T11:51:00.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 -- 172.21.15.177:0/1480239099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 msgr2=0x7f06540af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 --2- 172.21.15.177:0/1480239099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 0x7f06540af400 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0644009dc0 tx=0x7f064400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:00.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 -- 172.21.15.177:0/1480239099 shutdown_connections 2023-11-28T11:51:00.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 --2- 172.21.15.177:0/1480239099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065415cef0 0x7f06540af400 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 -- 172.21.15.177:0/1480239099 >> 172.21.15.177:0/1480239099 conn(0x7f06540d0000 msgr2=0x7f06540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 -- 172.21.15.177:0/1480239099 shutdown_connections 2023-11-28T11:51:00.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.087+0000 7f0659d58700 1 -- 172.21.15.177:0/1480239099 wait complete. 2023-11-28T11:51:00.092 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.088+0000 7f0659d58700 1 Processor -- start 2023-11-28T11:51:00.092 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.088+0000 7f0659d58700 1 -- start start 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0659d58700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 0x7f06540c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0659d58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06540cbdf0 0x7f065415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0659d58700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 0x7f06540ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0659d58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540c7860 con 0x7f06540cbdf0 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0659d58700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540b2e30 con 0x7f065415cef0 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0659d58700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06540b3110 con 0x7f065406ba90 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0653fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 0x7f06540ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 0x7f06540c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0653fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 0x7f06540ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40264/0 (socket says 172.21.15.177:40264) 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 0x7f06540c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48718/0 (socket says 172.21.15.177:48718) 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0653fff700 1 -- 172.21.15.177:0/2129231876 learned_addr learned my addr 172.21.15.177:0/2129231876 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:00.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0652ffd700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06540cbdf0 0x7f065415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 -- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 msgr2=0x7f06540ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 0x7f06540ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 -- 172.21.15.177:0/2129231876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06540cbdf0 msgr2=0x7f065415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06540cbdf0 0x7f065415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f06537fe700 1 -- 172.21.15.177:0/2129231876 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0644009a70 con 0x7f065406ba90 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.089+0000 7f0653fff700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 0x7f06540ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.090+0000 7f06537fe700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 0x7f06540c7320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0644018000 tx=0x7f0644004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.090+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644019600 con 0x7f065406ba90 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.090+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064401f040 con 0x7f065406ba90 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.090+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065415dff0 con 0x7f065406ba90 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.090+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440199c0 con 0x7f065406ba90 2023-11-28T11:51:00.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.090+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065415dd10 con 0x7f065406ba90 2023-11-28T11:51:00.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.091+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f064402d460 con 0x7f065406ba90 2023-11-28T11:51:00.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.091+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0644019e10 con 0x7f065406ba90 2023-11-28T11:51:00.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.091+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06540cad60 con 0x7f065406ba90 2023-11-28T11:51:00.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.093+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f064402d9e0 con 0x7f065406ba90 2023-11-28T11:51:00.160 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:00.159+0000 7f6e7a3cf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:51:00.161 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:00.159+0000 7fa453583200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:51:00.177 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.176+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9ca99a48-48b5-41bc-bade-5d9fdb9bef65", "id": 2} v 0) v1 -- 0x7f065406f440 con 0x7f065406ba90 2023-11-28T11:51:00.185 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.184+0000 7f064bfff700 1 -- 172.21.15.177:0/2129231876 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ca99a48-48b5-41bc-bade-5d9fdb9bef65", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f064402dc50 con 0x7f065406ba90 2023-11-28T11:51:00.186 INFO:teuthology.orchestra.run.smithi177.stdout:2 2023-11-28T11:51:00.187 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 msgr2=0x7f06540c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.187 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 0x7f06540c7320 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0644018000 tx=0x7f0644004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.187 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 shutdown_connections 2023-11-28T11:51:00.188 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f065415cef0 0x7f06540ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.188 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f065406ba90 0x7f06540c7320 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 --2- 172.21.15.177:0/2129231876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06540cbdf0 0x7f065415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 >> 172.21.15.177:0/2129231876 conn(0x7f06540d0000 msgr2=0x7f06540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 shutdown_connections 2023-11-28T11:51:00.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.185+0000 7f0659d58700 1 -- 172.21.15.177:0/2129231876 wait complete. 2023-11-28T11:51:00.198 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new c5f53b05-55f1-4c8f-aec4-40722b4132ad 3 2023-11-28T11:51:00.236 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:00.235+0000 7fa453583200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:51:00.239 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:00.238+0000 7f6e7a3cf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:51:00.312 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:00.310+0000 7fa453583200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:51:00.317 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:00.316+0000 7f6e7a3cf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:51:00.319 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 Processor -- start 2023-11-28T11:51:00.319 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 -- start start 2023-11-28T11:51:00.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 0x7f4328140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f432806f850 0x7f432806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432813ff00 con 0x7f43280cdc90 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43281404c0 con 0x7f432806f850 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.318+0000 7f432e7c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432815aa80 con 0x7f43280cd8c0 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 0x7f4328140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 0x7f4328140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48724/0 (socket says 172.21.15.177:48724) 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 -- 172.21.15.177:0/1752772749 learned_addr learned my addr 172.21.15.177:0/1752772749 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f432cd61700 1 --2- 172.21.15.177:0/1752772749 >> v2:172.21.15.144:3300/0 conn(0x7f432806f850 0x7f432806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 -- 172.21.15.177:0/1752772749 >> v2:172.21.15.144:3300/0 conn(0x7f432806f850 msgr2=0x7f432806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 --2- 172.21.15.177:0/1752772749 >> v2:172.21.15.144:3300/0 conn(0x7f432806f850 0x7f432806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 -- 172.21.15.177:0/1752772749 >> v1:172.21.15.144:6789/0 conn(0x7f43280cdc90 legacy=0x7f43280b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 -- 172.21.15.177:0/1752772749 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280b3040 con 0x7f43280cd8c0 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4327fff700 1 --2- 172.21.15.177:0/1752772749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 0x7f4328140950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4318006fd0 tx=0x7f4318009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed5dc199974b7dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4326ffd700 1 -- 172.21.15.177:0/1752772749 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318005e30 con 0x7f43280cd8c0 2023-11-28T11:51:00.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4326ffd700 1 -- 172.21.15.177:0/1752772749 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318004030 con 0x7f43280cd8c0 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.319+0000 7f4326ffd700 1 -- 172.21.15.177:0/1752772749 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180043e0 con 0x7f43280cd8c0 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 -- 172.21.15.177:0/1752772749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 msgr2=0x7f4328140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 --2- 172.21.15.177:0/1752772749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 0x7f4328140950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4318006fd0 tx=0x7f4318009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 -- 172.21.15.177:0/1752772749 shutdown_connections 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 --2- 172.21.15.177:0/1752772749 >> v2:172.21.15.144:3300/0 conn(0x7f432806f850 0x7f432806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 --2- 172.21.15.177:0/1752772749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43280cd8c0 0x7f4328140950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 -- 172.21.15.177:0/1752772749 >> 172.21.15.177:0/1752772749 conn(0x7f432806aed0 msgr2=0x7f432806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 -- 172.21.15.177:0/1752772749 shutdown_connections 2023-11-28T11:51:00.323 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.320+0000 7f432e7c4700 1 -- 172.21.15.177:0/1752772749 wait complete. 2023-11-28T11:51:00.325 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.321+0000 7f432e7c4700 1 Processor -- start 2023-11-28T11:51:00.325 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.321+0000 7f432e7c4700 1 -- start start 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432e7c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 0x7f4328159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432e7c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43280cd8c0 0x7f432806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432e7c4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43280cdc90 0x7f432813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432e7c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432815a160 con 0x7f43280cd8c0 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432e7c4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43281f1f60 con 0x7f43280cdc90 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432e7c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43281f2240 con 0x7f432806f850 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 0x7f4328159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f432cd61700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43280cdc90 0x7f432813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 0x7f4328159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48730/0 (socket says 172.21.15.177:48730) 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 -- 172.21.15.177:0/654026737 learned_addr learned my addr 172.21.15.177:0/654026737 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f43277fe700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43280cd8c0 0x7f432806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.326 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 -- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43280cdc90 msgr2=0x7f432813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43280cdc90 0x7f432813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 -- 172.21.15.177:0/654026737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43280cd8c0 msgr2=0x7f432806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43280cd8c0 0x7f432806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.322+0000 7f4327fff700 1 -- 172.21.15.177:0/654026737 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4318009500 con 0x7f432806f850 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f4327fff700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 0x7f4328159c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4318009c10 tx=0x7f4318004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318003c70 con 0x7f432806f850 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318003dd0 con 0x7f432806f850 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43280b35f0 con 0x7f432806f850 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431801c640 con 0x7f432806f850 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280b3310 con 0x7f432806f850 2023-11-28T11:51:00.327 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f431802c070 con 0x7f432806f850 2023-11-28T11:51:00.328 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.323+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f43180118b0 con 0x7f432806f850 2023-11-28T11:51:00.328 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.324+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4328140150 con 0x7f432806f850 2023-11-28T11:51:00.328 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.326+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f431802a060 con 0x7f432806f850 2023-11-28T11:51:00.423 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.422+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c5f53b05-55f1-4c8f-aec4-40722b4132ad", "id": 3} v 0) v1 -- 0x7f43281f2d20 con 0x7f432806f850 2023-11-28T11:51:00.431 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.430+0000 7f4324ff9700 1 -- 172.21.15.177:0/654026737 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c5f53b05-55f1-4c8f-aec4-40722b4132ad", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f43180340b0 con 0x7f432806f850 2023-11-28T11:51:00.431 INFO:teuthology.orchestra.run.smithi177.stdout:3 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 msgr2=0x7f4328159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 0x7f4328159c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4318009c10 tx=0x7f4318004750 comp rx=0 tx=0).stop 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 shutdown_connections 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43280cdc90 0x7f432813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f432806f850 0x7f4328159c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 --2- 172.21.15.177:0/654026737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43280cd8c0 0x7f432806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 >> 172.21.15.177:0/654026737 conn(0x7f432806aed0 msgr2=0x7f43280b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 shutdown_connections 2023-11-28T11:51:00.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.431+0000 7f432e7c4700 1 -- 172.21.15.177:0/654026737 wait complete. 2023-11-28T11:51:00.445 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 4f434f89-a39d-454a-9e0b-2b9fcbe3cce5 4 2023-11-28T11:51:00.449 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:00.447+0000 7fa453583200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:51:00.471 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:00.470+0000 7f6e7a3cf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:51:00.591 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.589+0000 7fa6c6da1700 1 Processor -- start 2023-11-28T11:51:00.591 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.589+0000 7fa6c6da1700 1 -- start start 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6c6da1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa6c006ba90 0x7fa6c00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6c6da1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c00cc5f0 con 0x7fa6c00cbdf0 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6c6da1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c015d470 con 0x7fa6c015cef0 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6c6da1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c015d750 con 0x7fa6c006ba90 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6bffff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa6c006ba90 0x7fa6c00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6c4b3d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa6c015cef0 0x7fa6c006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:44626/0 (socket says 172.21.15.177:44626) 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6c4b3d700 1 -- 172.21.15.177:0/117590116 learned_addr learned my addr 172.21.15.177:0/117590116 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6bf7fe700 1 -- 172.21.15.177:0/117590116 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469074782 0 0) 0x7fa6c015d470 con 0x7fa6c015cef0 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.590+0000 7fa6bf7fe700 1 -- 172.21.15.177:0/117590116 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b00035b0 con 0x7fa6c015cef0 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bffff700 1 -- 172.21.15.177:0/117590116 >> v1:172.21.15.177:6789/0 conn(0x7fa6c015cef0 legacy=0x7fa6c006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.593 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bffff700 1 -- 172.21.15.177:0/117590116 >> v1:172.21.15.144:6789/0 conn(0x7fa6c00cbdf0 legacy=0x7fa6c00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bffff700 1 -- 172.21.15.177:0/117590116 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c015da30 con 0x7fa6c006ba90 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bffff700 1 --2- 172.21.15.177:0/117590116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c00af400 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009840 tx=0x7fa6ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad33cabb91c9f6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bf7fe700 1 -- 172.21.15.177:0/117590116 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011830 con 0x7fa6c006ba90 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bf7fe700 1 -- 172.21.15.177:0/117590116 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac011990 con 0x7fa6c006ba90 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6bf7fe700 1 -- 172.21.15.177:0/117590116 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011d20 con 0x7fa6c006ba90 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 -- 172.21.15.177:0/117590116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 msgr2=0x7fa6c00af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 --2- 172.21.15.177:0/117590116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c00af400 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009840 tx=0x7fa6ac009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 -- 172.21.15.177:0/117590116 shutdown_connections 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 --2- 172.21.15.177:0/117590116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c00af400 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 -- 172.21.15.177:0/117590116 >> 172.21.15.177:0/117590116 conn(0x7fa6c00d0000 msgr2=0x7fa6c00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 -- 172.21.15.177:0/117590116 shutdown_connections 2023-11-28T11:51:00.594 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.591+0000 7fa6c6da1700 1 -- 172.21.15.177:0/117590116 wait complete. 2023-11-28T11:51:00.598 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.593+0000 7fa6c6da1700 1 Processor -- start 2023-11-28T11:51:00.598 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.593+0000 7fa6c6da1700 1 -- start start 2023-11-28T11:51:00.598 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.593+0000 7fa6c6da1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c0153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.598 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.593+0000 7fa6c6da1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c00cbdf0 0x7fa6c00c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.598 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.593+0000 7fa6c6da1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 0x7fa6c00b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.598 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c6da1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00ad680 con 0x7fa6c00cbdf0 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c6da1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00ad0c0 con 0x7fa6c006ba90 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c6da1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00ad3a0 con 0x7fa6c015cef0 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c533e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 0x7fa6c00b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c0153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c533e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 0x7fa6c00b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48746/0 (socket says 172.21.15.177:48746) 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c0153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40284/0 (socket says 172.21.15.177:40284) 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c533e700 1 -- 172.21.15.177:0/3684071092 learned_addr learned my addr 172.21.15.177:0/3684071092 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:00.599 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6bffff700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c00cbdf0 0x7fa6c00c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.601 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 -- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 msgr2=0x7fa6c00b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.601 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 0x7fa6c00b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.602 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 -- 172.21.15.177:0/3684071092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c00cbdf0 msgr2=0x7fa6c00c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.602 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c00cbdf0 0x7fa6c00c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.602 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 -- 172.21.15.177:0/3684071092 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0094a0 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c533e700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 0x7fa6c00b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c4b3d700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c0153230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c430 tx=0x7fa6b400c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4005d70 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40070d0 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4007460 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c015dff0 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c015dd10 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.594+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa6b4010070 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.595+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fa6b4012400 con 0x7fa6c006ba90 2023-11-28T11:51:00.603 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.595+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c00cad60 con 0x7fa6c006ba90 2023-11-28T11:51:00.604 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.598+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa6b4017070 con 0x7fa6c006ba90 2023-11-28T11:51:00.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.680+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4f434f89-a39d-454a-9e0b-2b9fcbe3cce5", "id": 4} v 0) v1 -- 0x7fa6c006ae00 con 0x7fa6c006ba90 2023-11-28T11:51:00.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.688+0000 7fa6bd7fa700 1 -- 172.21.15.177:0/3684071092 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f434f89-a39d-454a-9e0b-2b9fcbe3cce5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa6b40125c0 con 0x7fa6c006ba90 2023-11-28T11:51:00.690 INFO:teuthology.orchestra.run.smithi177.stdout:4 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 msgr2=0x7fa6c0153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c0153230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c430 tx=0x7fa6b400c740 comp rx=0 tx=0).stop 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 shutdown_connections 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6c015cef0 0x7fa6c00b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6c006ba90 0x7fa6c0153230 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 --2- 172.21.15.177:0/3684071092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6c00cbdf0 0x7fa6c00c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 >> 172.21.15.177:0/3684071092 conn(0x7fa6c00d0000 msgr2=0x7fa6c006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 shutdown_connections 2023-11-28T11:51:00.691 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.689+0000 7fa6c6da1700 1 -- 172.21.15.177:0/3684071092 wait complete. 2023-11-28T11:51:00.703 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 3ba5d415-e46a-4d3e-b94f-86521b067ea1 5 2023-11-28T11:51:00.825 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.824+0000 7f4592e0f700 1 Processor -- start 2023-11-28T11:51:00.826 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.824+0000 7f4592e0f700 1 -- start start 2023-11-28T11:51:00.828 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f4592e0f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f458c06e9f0 0x7f458c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.828 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f4592e0f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f458c06d0a0 0x7f458c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.828 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f4592e0f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c06ae60 con 0x7f458c06b830 2023-11-28T11:51:00.828 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f4592e0f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c06a270 con 0x7f458c06e9f0 2023-11-28T11:51:00.828 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f4592e0f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c06a530 con 0x7f458c06d0a0 2023-11-28T11:51:00.828 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f45913ac700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f458c06d0a0 0x7f458c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f458bfff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f458c06b830 0x7f458c06c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:44640/0 (socket says 172.21.15.177:44640) 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f45913ac700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f458c06d0a0 0x7f458c0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40294/0 (socket says 172.21.15.177:40294) 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f458bfff700 1 -- 172.21.15.177:0/3982376157 learned_addr learned my addr 172.21.15.177:0/3982376157 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f4590bab700 1 --2- 172.21.15.177:0/3982376157 >> v2:172.21.15.144:3300/0 conn(0x7f458c06e9f0 0x7f458c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f458b7fe700 1 -- 172.21.15.177:0/3982376157 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520039744 0 0) 0x7f458c06ae60 con 0x7f458c06b830 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f458b7fe700 1 -- 172.21.15.177:0/3982376157 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f457c0035b0 con 0x7f458c06b830 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f45913ac700 1 -- 172.21.15.177:0/3982376157 >> v2:172.21.15.144:3300/0 conn(0x7f458c06e9f0 msgr2=0x7f458c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f45913ac700 1 --2- 172.21.15.177:0/3982376157 >> v2:172.21.15.144:3300/0 conn(0x7f458c06e9f0 0x7f458c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f45913ac700 1 -- 172.21.15.177:0/3982376157 >> v1:172.21.15.177:6789/0 conn(0x7f458c06b830 legacy=0x7f458c06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.825+0000 7f45913ac700 1 -- 172.21.15.177:0/3982376157 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c06a810 con 0x7f458c06d0a0 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f45913ac700 1 --2- 172.21.15.177:0/3982376157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06d0a0 0x7f458c0ca240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f458c070490 tx=0x7f458400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d1bb2e2d6c611e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f458b7fe700 1 -- 172.21.15.177:0/3982376157 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400e070 con 0x7f458c06d0a0 2023-11-28T11:51:00.829 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f458b7fe700 1 -- 172.21.15.177:0/3982376157 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400bb20 con 0x7f458c06d0a0 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f458b7fe700 1 -- 172.21.15.177:0/3982376157 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584012430 con 0x7f458c06d0a0 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 -- 172.21.15.177:0/3982376157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06d0a0 msgr2=0x7f458c0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 --2- 172.21.15.177:0/3982376157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06d0a0 0x7f458c0ca240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f458c070490 tx=0x7f458400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 -- 172.21.15.177:0/3982376157 shutdown_connections 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 --2- 172.21.15.177:0/3982376157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06d0a0 0x7f458c0ca240 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 --2- 172.21.15.177:0/3982376157 >> v2:172.21.15.144:3300/0 conn(0x7f458c06e9f0 0x7f458c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 -- 172.21.15.177:0/3982376157 >> 172.21.15.177:0/3982376157 conn(0x7f458c0b12b0 msgr2=0x7f458c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 -- 172.21.15.177:0/3982376157 shutdown_connections 2023-11-28T11:51:00.830 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.826+0000 7f4592e0f700 1 -- 172.21.15.177:0/3982376157 wait complete. 2023-11-28T11:51:00.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.827+0000 7f4592e0f700 1 Processor -- start 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 -- start start 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06b830 0x7f458c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 0x7f458c0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f458c06e9f0 0x7f458c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1f2fc0 con 0x7f458c06e9f0 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0ae140 con 0x7f458c06b830 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.828+0000 7f4592e0f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0ae3a0 con 0x7f458c06d0a0 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f458bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 0x7f458c0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06b830 0x7f458c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f458bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 0x7f458c0adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48756/0 (socket says 172.21.15.177:48756) 2023-11-28T11:51:00.833 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f458bfff700 1 -- 172.21.15.177:0/4032380300 learned_addr learned my addr 172.21.15.177:0/4032380300 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f45913ac700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f458c06e9f0 0x7f458c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 -- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 msgr2=0x7f458c0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 0x7f458c0adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 -- 172.21.15.177:0/4032380300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f458c06e9f0 msgr2=0x7f458c152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f458c06e9f0 0x7f458c152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 -- 172.21.15.177:0/4032380300 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4584009a70 con 0x7f458c06b830 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f458bfff700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 0x7f458c0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4590bab700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06b830 0x7f458c1f6630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f458000e900 tx=0x7f458000ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580013680 con 0x7f458c06b830 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45800137e0 con 0x7f458c06b830 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580013b50 con 0x7f458c06b830 2023-11-28T11:51:00.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c06edc0 con 0x7f458c06b830 2023-11-28T11:51:00.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c06aaf0 con 0x7f458c06b830 2023-11-28T11:51:00.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4580019030 con 0x7f458c06b830 2023-11-28T11:51:00.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.829+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f4580013cb0 con 0x7f458c06b830 2023-11-28T11:51:00.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.830+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c06ef50 con 0x7f458c06b830 2023-11-28T11:51:00.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.832+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f458003ab00 con 0x7f458c06b830 2023-11-28T11:51:00.839 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:00.838+0000 7fa453583200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:51:00.893 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:00.891+0000 7f6e7a3cf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:51:00.928 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.927+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ba5d415-e46a-4d3e-b94f-86521b067ea1", "id": 5} v 0) v1 -- 0x7f458c1f3230 con 0x7f458c06b830 2023-11-28T11:51:00.938 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.937+0000 7f45897fa700 1 -- 172.21.15.177:0/4032380300 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ba5d415-e46a-4d3e-b94f-86521b067ea1", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4580029070 con 0x7f458c06b830 2023-11-28T11:51:00.938 INFO:teuthology.orchestra.run.smithi177.stdout:5 2023-11-28T11:51:00.939 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06b830 msgr2=0x7f458c1f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:00.939 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06b830 0x7f458c1f6630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f458000e900 tx=0x7f458000ec10 comp rx=0 tx=0).stop 2023-11-28T11:51:00.939 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 shutdown_connections 2023-11-28T11:51:00.939 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f458c06e9f0 0x7f458c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.939 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f458c06b830 0x7f458c1f6630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.939 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 --2- 172.21.15.177:0/4032380300 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f458c06d0a0 0x7f458c0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:00.940 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 >> 172.21.15.177:0/4032380300 conn(0x7f458c0b12b0 msgr2=0x7f458c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:00.940 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 shutdown_connections 2023-11-28T11:51:00.940 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:00.938+0000 7f4592e0f700 1 -- 172.21.15.177:0/4032380300 wait complete. 2023-11-28T11:51:00.952 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 807086dc-2ec0-487a-af31-01d27a3db051 6 2023-11-28T11:51:01.072 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.070+0000 7f4988544700 1 Processor -- start 2023-11-28T11:51:01.072 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.070+0000 7f4988544700 1 -- start start 2023-11-28T11:51:01.074 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.071+0000 7f4988544700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f49801658c0 0x7f4980167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.074 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.071+0000 7f4988544700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498015d060 con 0x7f498006dee0 2023-11-28T11:51:01.074 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.071+0000 7f4988544700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498015d340 con 0x7f4980071040 2023-11-28T11:51:01.074 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.071+0000 7f4988544700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4980161550 con 0x7f49801658c0 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49862e0700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f4980071040 0x7f4980071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:44650/0 (socket says 172.21.15.177:44650) 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49862e0700 1 -- 172.21.15.177:0/3294588433 learned_addr learned my addr 172.21.15.177:0/3294588433 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f4986ae1700 1 --2- 172.21.15.177:0/3294588433 >> v2:172.21.15.177:3300/0 conn(0x7f49801658c0 0x7f4980167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49852de700 1 -- 172.21.15.177:0/3294588433 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528066353 0 0) 0x7f498015d340 con 0x7f4980071040 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49852de700 1 -- 172.21.15.177:0/3294588433 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f496c0035b0 con 0x7f4980071040 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f4986ae1700 1 -- 172.21.15.177:0/3294588433 >> v1:172.21.15.177:6789/0 conn(0x7f4980071040 legacy=0x7f4980071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f4986ae1700 1 -- 172.21.15.177:0/3294588433 >> v1:172.21.15.144:6789/0 conn(0x7f498006dee0 legacy=0x7f498006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f4986ae1700 1 -- 172.21.15.177:0/3294588433 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498015d620 con 0x7f49801658c0 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f4986ae1700 1 --2- 172.21.15.177:0/3294588433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49801658c0 0x7f4980167ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4978009d90 tx=0x7f497800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c411215d712d7c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49852de700 1 -- 172.21.15.177:0/3294588433 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800e070 con 0x7f49801658c0 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49852de700 1 -- 172.21.15.177:0/3294588433 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497800bb20 con 0x7f49801658c0 2023-11-28T11:51:01.075 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.072+0000 7f49852de700 1 -- 172.21.15.177:0/3294588433 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978012430 con 0x7f49801658c0 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 -- 172.21.15.177:0/3294588433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49801658c0 msgr2=0x7f4980167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 --2- 172.21.15.177:0/3294588433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49801658c0 0x7f4980167ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4978009d90 tx=0x7f497800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 -- 172.21.15.177:0/3294588433 shutdown_connections 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 --2- 172.21.15.177:0/3294588433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49801658c0 0x7f4980167ca0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 -- 172.21.15.177:0/3294588433 >> 172.21.15.177:0/3294588433 conn(0x7f49800d0000 msgr2=0x7f49800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 -- 172.21.15.177:0/3294588433 shutdown_connections 2023-11-28T11:51:01.076 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.073+0000 7f4988544700 1 -- 172.21.15.177:0/3294588433 wait complete. 2023-11-28T11:51:01.079 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.074+0000 7f4988544700 1 Processor -- start 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.074+0000 7f4988544700 1 -- start start 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4988544700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f498006dee0 0x7f49801454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4988544700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4980071040 0x7f4980142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4988544700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49801658c0 0x7f4980156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4988544700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4980157400 con 0x7f49801658c0 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4988544700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49801459f0 con 0x7f498006dee0 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4988544700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49800ace90 con 0x7f4980071040 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4985adf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4980071040 0x7f4980142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f498006dee0 0x7f49801454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4985adf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4980071040 0x7f4980142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:48768/0 (socket says 172.21.15.177:48768) 2023-11-28T11:51:01.080 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4985adf700 1 -- 172.21.15.177:0/702891769 learned_addr learned my addr 172.21.15.177:0/702891769 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f4986ae1700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49801658c0 0x7f4980156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 -- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4980071040 msgr2=0x7f4980142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4980071040 0x7f4980142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 -- 172.21.15.177:0/702891769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49801658c0 msgr2=0x7f4980156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49801658c0 0x7f4980156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 -- 172.21.15.177:0/702891769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978009a70 con 0x7f498006dee0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.075+0000 7f49862e0700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f498006dee0 0x7f49801454b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f497c00c430 tx=0x7f497c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c005b70 con 0x7f498006dee0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c005cd0 con 0x7f498006dee0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c007310 con 0x7f498006dee0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498015ccd0 con 0x7f498006dee0 2023-11-28T11:51:01.081 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498015d900 con 0x7f498006dee0 2023-11-28T11:51:01.082 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f497c010070 con 0x7f498006dee0 2023-11-28T11:51:01.082 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.076+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f497c005e40 con 0x7f498006dee0 2023-11-28T11:51:01.082 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.077+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498006a270 con 0x7f498006dee0 2023-11-28T11:51:01.082 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.079+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f497c017070 con 0x7f498006dee0 2023-11-28T11:51:01.163 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.161+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "807086dc-2ec0-487a-af31-01d27a3db051", "id": 6} v 0) v1 -- 0x7f498006c370 con 0x7f498006dee0 2023-11-28T11:51:01.171 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.170+0000 7f4976ffd700 1 -- 172.21.15.177:0/702891769 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "807086dc-2ec0-487a-af31-01d27a3db051", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f497c012400 con 0x7f498006dee0 2023-11-28T11:51:01.171 INFO:teuthology.orchestra.run.smithi177.stdout:6 2023-11-28T11:51:01.172 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f498006dee0 msgr2=0x7f49801454b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.172 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f498006dee0 0x7f49801454b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f497c00c430 tx=0x7f497c00c740 comp rx=0 tx=0).stop 2023-11-28T11:51:01.172 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 shutdown_connections 2023-11-28T11:51:01.172 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49801658c0 0x7f4980156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.173 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f498006dee0 0x7f49801454b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.173 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 --2- 172.21.15.177:0/702891769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4980071040 0x7f4980142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.173 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 >> 172.21.15.177:0/702891769 conn(0x7f49800d0000 msgr2=0x7f49800af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:01.173 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 shutdown_connections 2023-11-28T11:51:01.173 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.171+0000 7f4988544700 1 -- 172.21.15.177:0/702891769 wait complete. 2023-11-28T11:51:01.184 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 12c21e8c-9c83-4d7f-8baf-f0154e97b9ca 7 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.310+0000 7f821e8bf700 1 Processor -- start 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.310+0000 7f821e8bf700 1 -- start start 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.311+0000 7f821e8bf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f82180ce520 0x7f82180cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.311+0000 7f821e8bf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f82180b0670 0x7f82180b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.311+0000 7f821e8bf700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82180b1010 con 0x7f82180cacb0 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.311+0000 7f821e8bf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821806c2f0 con 0x7f82180b0670 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.311+0000 7f821e8bf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82180cd740 con 0x7f82180ce520 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.311+0000 7f82177fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f82180ce520 0x7f82180cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f82180ce520 0x7f82180cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40316/0 (socket says 172.21.15.177:40316) 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 -- 172.21.15.177:0/2248833677 learned_addr learned my addr 172.21.15.177:0/2248833677 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f821ce5c700 1 --2- 172.21.15.177:0/2248833677 >> v2:172.21.15.144:3300/0 conn(0x7f82180b0670 0x7f82180b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f8216ffd700 1 -- 172.21.15.177:0/2248833677 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768552957 0 0) 0x7f82180b1010 con 0x7f82180cacb0 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f8216ffd700 1 -- 172.21.15.177:0/2248833677 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82040035b0 con 0x7f82180cacb0 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 -- 172.21.15.177:0/2248833677 >> v2:172.21.15.144:3300/0 conn(0x7f82180b0670 msgr2=0x7f82180b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 --2- 172.21.15.177:0/2248833677 >> v2:172.21.15.144:3300/0 conn(0x7f82180b0670 0x7f82180b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 -- 172.21.15.177:0/2248833677 >> v1:172.21.15.177:6789/0 conn(0x7f82180cacb0 legacy=0x7f82180cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 -- 172.21.15.177:0/2248833677 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821806b9c0 con 0x7f82180ce520 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.312+0000 7f82177fe700 1 --2- 172.21.15.177:0/2248833677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180ce520 0x7f82180cb080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f82180c9be0 tx=0x7f820800d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfb8331ded6e2dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f8216ffd700 1 -- 172.21.15.177:0/2248833677 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208010070 con 0x7f82180ce520 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f8216ffd700 1 -- 172.21.15.177:0/2248833677 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820800d910 con 0x7f82180ce520 2023-11-28T11:51:01.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f8216ffd700 1 -- 172.21.15.177:0/2248833677 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820800dcc0 con 0x7f82180ce520 2023-11-28T11:51:01.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 -- 172.21.15.177:0/2248833677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180ce520 msgr2=0x7f82180cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 --2- 172.21.15.177:0/2248833677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180ce520 0x7f82180cb080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f82180c9be0 tx=0x7f820800d330 comp rx=0 tx=0).stop 2023-11-28T11:51:01.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 -- 172.21.15.177:0/2248833677 shutdown_connections 2023-11-28T11:51:01.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 --2- 172.21.15.177:0/2248833677 >> v2:172.21.15.144:3300/0 conn(0x7f82180b0670 0x7f82180b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 --2- 172.21.15.177:0/2248833677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180ce520 0x7f82180cb080 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 -- 172.21.15.177:0/2248833677 >> 172.21.15.177:0/2248833677 conn(0x7f82180cb670 msgr2=0x7f82180cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:01.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 -- 172.21.15.177:0/2248833677 shutdown_connections 2023-11-28T11:51:01.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.313+0000 7f821e8bf700 1 -- 172.21.15.177:0/2248833677 wait complete. 2023-11-28T11:51:01.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.315+0000 7f821e8bf700 1 Processor -- start 2023-11-28T11:51:01.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.315+0000 7f821e8bf700 1 -- start start 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821e8bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82180b0670 0x7f8218206eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821e8bf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180cacb0 0x7f821816a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821e8bf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82180c2500 0x7f821816a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821e8bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8218087be0 con 0x7f82180b0670 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821e8bf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82182073f0 con 0x7f82180cacb0 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821e8bf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82180c5210 con 0x7f82180c2500 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f82177fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180cacb0 0x7f821816a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f8217fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82180b0670 0x7f8218206eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f82177fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180cacb0 0x7f821816a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40322/0 (socket says 172.21.15.177:40322) 2023-11-28T11:51:01.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f82177fe700 1 -- 172.21.15.177:0/1586016428 learned_addr learned my addr 172.21.15.177:0/1586016428 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-28T11:51:01.319 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f821ce5c700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82180c2500 0x7f821816a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.320 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f8217fff700 1 -- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82180c2500 msgr2=0x7f821816a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.320 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f8217fff700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82180c2500 0x7f821816a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.320 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f8217fff700 1 -- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180cacb0 msgr2=0x7f821816a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.320 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.316+0000 7f8217fff700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180cacb0 0x7f821816a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.320 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8217fff700 1 -- 172.21.15.177:0/1586016428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080086d0 con 0x7f82180b0670 2023-11-28T11:51:01.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8217fff700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82180b0670 0x7f8218206eb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f821000cbe0 tx=0x7f821000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:01.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210003c00 con 0x7f82180b0670 2023-11-28T11:51:01.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8210003d60 con 0x7f82180b0670 2023-11-28T11:51:01.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821000ab90 con 0x7f82180b0670 2023-11-28T11:51:01.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821806bf80 con 0x7f82180b0670 2023-11-28T11:51:01.321 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821806bca0 con 0x7f82180b0670 2023-11-28T11:51:01.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8210010050 con 0x7f82180b0670 2023-11-28T11:51:01.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.317+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8210012420 con 0x7f82180b0670 2023-11-28T11:51:01.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.318+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821806c780 con 0x7f82180b0670 2023-11-28T11:51:01.322 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.320+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f82100050d0 con 0x7f82180b0670 2023-11-28T11:51:01.394 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:01.392+0000 7fa453583200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:51:01.425 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.424+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12c21e8c-9c83-4d7f-8baf-f0154e97b9ca", "id": 7} v 0) v1 -- 0x7f82180745c0 con 0x7f82180b0670 2023-11-28T11:51:01.433 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.432+0000 7f8214ff9700 1 -- 172.21.15.177:0/1586016428 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12c21e8c-9c83-4d7f-8baf-f0154e97b9ca", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8210005270 con 0x7f82180b0670 2023-11-28T11:51:01.433 INFO:teuthology.orchestra.run.smithi177.stdout:7 2023-11-28T11:51:01.435 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82180b0670 msgr2=0x7f8218206eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.435 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82180b0670 0x7f8218206eb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f821000cbe0 tx=0x7f821000cef0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.435 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 shutdown_connections 2023-11-28T11:51:01.435 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82180b0670 0x7f8218206eb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.435 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82180c2500 0x7f821816a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.436 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 --2- 172.21.15.177:0/1586016428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82180cacb0 0x7f821816a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.436 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 >> 172.21.15.177:0/1586016428 conn(0x7f82180cb670 msgr2=0x7f82181f1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:01.436 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 shutdown_connections 2023-11-28T11:51:01.436 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-28T11:51:01.433+0000 7f821e8bf700 1 -- 172.21.15.177:0/1586016428 wait complete. 2023-11-28T11:51:01.447 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T11:51:01.447 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T11:51:01.450 INFO:tasks.ceph.osd.0:Started 2023-11-28T11:51:01.450 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T11:51:01.451 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T11:51:01.453 INFO:tasks.ceph.osd.1:Started 2023-11-28T11:51:01.453 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T11:51:01.453 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T11:51:01.455 INFO:tasks.ceph.osd.2:Started 2023-11-28T11:51:01.456 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T11:51:01.456 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T11:51:01.461 INFO:tasks.ceph.osd.3:Started 2023-11-28T11:51:01.461 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T11:51:01.461 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T11:51:01.464 INFO:tasks.ceph.osd.4:Started 2023-11-28T11:51:01.465 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T11:51:01.465 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T11:51:01.467 INFO:tasks.ceph.osd.5:Started 2023-11-28T11:51:01.467 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T11:51:01.467 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T11:51:01.468 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:01.466+0000 7fa453583200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:51:01.469 INFO:tasks.ceph.osd.6:Started 2023-11-28T11:51:01.470 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T11:51:01.470 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T11:51:01.476 INFO:tasks.ceph.osd.7:Started 2023-11-28T11:51:01.476 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:01.479 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:01.478+0000 7f6e7a3cf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:51:01.575 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:01.573+0000 7f6e7a3cf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:51:01.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.627+0000 7f0bee8e0700 1 Processor -- start 2023-11-28T11:51:01.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.627+0000 7f0bee8e0700 1 -- start start 2023-11-28T11:51:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0bee8e0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0be80ae9d0 0x7f0be815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0bee8e0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0be806a5e0 0x7f0be806a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0bee8e0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80aee40 con 0x7f0be806e500 2023-11-28T11:51:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0bee8e0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be815bf10 con 0x7f0be80ae9d0 2023-11-28T11:51:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0bee8e0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be806c930 con 0x7f0be806a5e0 2023-11-28T11:51:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0be80ae9d0 0x7f0be815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0be80ae9d0 0x7f0be815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33776/0 (socket says 172.21.15.144:33776) 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 -- 172.21.15.144:0/919870750 learned_addr learned my addr 172.21.15.144:0/919870750 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0bece7d700 1 --2- 172.21.15.144:0/919870750 >> v2:172.21.15.177:3300/0 conn(0x7f0be806a5e0 0x7f0be806a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 -- 172.21.15.144:0/919870750 >> v2:172.21.15.177:3300/0 conn(0x7f0be806a5e0 msgr2=0x7f0be806a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 --2- 172.21.15.144:0/919870750 >> v2:172.21.15.177:3300/0 conn(0x7f0be806a5e0 0x7f0be806a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 -- 172.21.15.144:0/919870750 >> v1:172.21.15.177:6789/0 conn(0x7f0be806e500 legacy=0x7f0be806e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 -- 172.21.15.144:0/919870750 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be806cc10 con 0x7f0be80ae9d0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be77fe700 1 --2- 172.21.15.144:0/919870750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be815b9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0be8072160 tx=0x7f0bdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835369f0a5550345 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be6ffd700 1 -- 172.21.15.144:0/919870750 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc011830 con 0x7f0be80ae9d0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be6ffd700 1 -- 172.21.15.144:0/919870750 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc011990 con 0x7f0be80ae9d0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.628+0000 7f0be6ffd700 1 -- 172.21.15.144:0/919870750 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc011d20 con 0x7f0be80ae9d0 2023-11-28T11:51:01.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 -- 172.21.15.144:0/919870750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 msgr2=0x7f0be815b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/919870750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be815b9d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0be8072160 tx=0x7f0bdc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 -- 172.21.15.144:0/919870750 shutdown_connections 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/919870750 >> v2:172.21.15.177:3300/0 conn(0x7f0be806a5e0 0x7f0be806a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/919870750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be815b9d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 -- 172.21.15.144:0/919870750 >> 172.21.15.144:0/919870750 conn(0x7f0be80b12b0 msgr2=0x7f0be80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 -- 172.21.15.144:0/919870750 shutdown_connections 2023-11-28T11:51:01.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.629+0000 7f0bee8e0700 1 -- 172.21.15.144:0/919870750 wait complete. 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.631+0000 7f0bee8e0700 1 Processor -- start 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.631+0000 7f0bee8e0700 1 -- start start 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bee8e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be806a5e0 0x7f0be8153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bee8e0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be806e500 0x7f0be8155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bee8e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be81567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bee8e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8154580 con 0x7f0be80ae9d0 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bee8e0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8154860 con 0x7f0be806e500 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bee8e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8154b40 con 0x7f0be806a5e0 2023-11-28T11:51:01.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bece7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be81567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bece7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be81567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33784/0 (socket says 172.21.15.144:33784) 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0bece7d700 1 -- 172.21.15.144:0/3854773608 learned_addr learned my addr 172.21.15.144:0/3854773608 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0be77fe700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be806e500 0x7f0be8155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.632+0000 7f0be7fff700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be806a5e0 0x7f0be8153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bece7d700 1 -- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be806a5e0 msgr2=0x7f0be8153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bece7d700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be806a5e0 0x7f0be8153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bece7d700 1 -- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be806e500 msgr2=0x7f0be8155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bece7d700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be806e500 0x7f0be8155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bece7d700 1 -- 172.21.15.144:0/3854773608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009550 con 0x7f0be80ae9d0 2023-11-28T11:51:01.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bece7d700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be81567f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0be000efc0 tx=0x7f0be000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000cad0 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be000f040 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be806cef0 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be000ce70 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be80664c0 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f0be0014480 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.633+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0be0014de0 con 0x7f0be80ae9d0 2023-11-28T11:51:01.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.634+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be815ac80 con 0x7f0be80ae9d0 2023-11-28T11:51:01.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.636+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0be0018020 con 0x7f0be80ae9d0 2023-11-28T11:51:01.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.705+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be8065fc0 con 0x7f0be80ae9d0 2023-11-28T11:51:01.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.706+0000 7f0be4ff9700 1 -- 172.21.15.144:0/3854773608 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0be0014840 con 0x7f0be80ae9d0 2023-11-28T11:51:01.708 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:01.708 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":10,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:01.426199+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 msgr2=0x7f0be81567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be81567f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0be000efc0 tx=0x7f0be000c0c0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 shutdown_connections 2023-11-28T11:51:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be80ae9d0 0x7f0be81567f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0be806a5e0 0x7f0be8153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 --2- 172.21.15.144:0/3854773608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0be806e500 0x7f0be8155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:01.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 >> 172.21.15.144:0/3854773608 conn(0x7f0be80b12b0 msgr2=0x7f0be80b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:01.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 shutdown_connections 2023-11-28T11:51:01.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:01.707+0000 7f0bee8e0700 1 -- 172.21.15.144:0/3854773608 wait complete. 2023-11-28T11:51:01.721 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T11:51:01.721 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T11:51:01.830 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T11:51:01.828+0000 7f0405cc2640 -1 Falling back to public interface 2023-11-28T11:51:01.841 INFO:tasks.ceph.osd.2.smithi144.stderr:2023-11-28T11:51:01.839+0000 7fa9e8740640 -1 Falling back to public interface 2023-11-28T11:51:02.068 INFO:tasks.ceph.osd.1.smithi144.stderr:2023-11-28T11:51:02.066+0000 7feae6d0f640 -1 Falling back to public interface 2023-11-28T11:51:02.088 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T11:51:02.087+0000 7fb21a6ac640 -1 Falling back to public interface 2023-11-28T11:51:02.093 INFO:tasks.ceph.osd.0.smithi144.stderr:2023-11-28T11:51:02.091+0000 7f0da9b6e640 -1 Falling back to public interface 2023-11-28T11:51:02.093 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T11:51:02.092+0000 7f4845171640 -1 Falling back to public interface 2023-11-28T11:51:02.094 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T11:51:02.092+0000 7f7d199a7640 -1 Falling back to public interface 2023-11-28T11:51:02.095 INFO:tasks.ceph.osd.3.smithi144.stderr:2023-11-28T11:51:02.093+0000 7fdcb2c9c640 -1 Falling back to public interface 2023-11-28T11:51:02.185 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:02.183+0000 7fa453583200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:51:02.255 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:02.254+0000 7f6e7a3cf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:51:02.262 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:02.260+0000 7fa453583200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:51:02.332 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:02.331+0000 7f6e7a3cf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:51:02.402 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:02.400+0000 7fa453583200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:51:02.472 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:02.470+0000 7f6e7a3cf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:51:02.528 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:02.644 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.642+0000 7fea64b4d700 1 Processor -- start 2023-11-28T11:51:02.644 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.642+0000 7fea64b4d700 1 -- start start 2023-11-28T11:51:02.649 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea64b4d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea6006dee0 0x7fea6006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:02.649 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea64b4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea601609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:02.649 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea64b4d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6015d060 con 0x7fea60071040 2023-11-28T11:51:02.649 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea64b4d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6015d340 con 0x7fea6006dee0 2023-11-28T11:51:02.649 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea64b4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6015d620 con 0x7fea60160540 2023-11-28T11:51:02.649 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5dd9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea6006dee0 0x7fea6006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea601609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5dd9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea6006dee0 0x7fea6006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36062/0 (socket says 172.21.15.144:36062) 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2923992825 learned_addr learned my addr 172.21.15.144:0/2923992825 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea601609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:45872/0 (socket says 172.21.15.144:45872) 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 msgr2=0x7fea601609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5dd9b700 1 --2- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea601609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2923992825 >> v1:172.21.15.144:6789/0 conn(0x7fea60071040 legacy=0x7fea60071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2923992825 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6015d900 con 0x7fea6006dee0 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea5ed9d700 1 --2- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea601609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:02.650 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.643+0000 7fea5d59a700 1 -- 172.21.15.144:0/2923992825 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573386196 0 0) 0x7fea6015d060 con 0x7fea60071040 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea5dd9b700 1 --2- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea6006e330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fea500098f0 tx=0x7fea50009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60854cde9cdfbf11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea5d59a700 1 -- 172.21.15.144:0/2923992825 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50011830 con 0x7fea6006dee0 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea5d59a700 1 -- 172.21.15.144:0/2923992825 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea50011990 con 0x7fea6006dee0 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea5d59a700 1 -- 172.21.15.144:0/2923992825 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50011d20 con 0x7fea6006dee0 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea64b4d700 1 -- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 msgr2=0x7fea6006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea64b4d700 1 --2- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea6006e330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fea500098f0 tx=0x7fea50009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea64b4d700 1 -- 172.21.15.144:0/2923992825 shutdown_connections 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea64b4d700 1 --2- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea601609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea64b4d700 1 --2- 172.21.15.144:0/2923992825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea6006e330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.644+0000 7fea64b4d700 1 -- 172.21.15.144:0/2923992825 >> 172.21.15.144:0/2923992825 conn(0x7fea600d0000 msgr2=0x7fea600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:02.651 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.645+0000 7fea64b4d700 1 -- 172.21.15.144:0/2923992825 shutdown_connections 2023-11-28T11:51:02.652 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.645+0000 7fea64b4d700 1 -- 172.21.15.144:0/2923992825 wait complete. 2023-11-28T11:51:02.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.646+0000 7fea64b4d700 1 Processor -- start 2023-11-28T11:51:02.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.646+0000 7fea64b4d700 1 -- start start 2023-11-28T11:51:02.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea64b4d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea60158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:02.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea64b4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 0x7fea6014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:02.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea64b4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea6014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea64b4d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea600b3380 con 0x7fea60071040 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea64b4d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6014f970 con 0x7fea6006dee0 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea64b4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea600b5d50 con 0x7fea60160540 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 0x7fea6014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 0x7fea6014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33794/0 (socket says 172.21.15.144:33794) 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2988547660 learned_addr learned my addr 172.21.15.144:0/2988547660 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5e59c700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea60158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:02.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5ed9d700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea6014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:02.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 msgr2=0x7fea6014f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:02.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea6014f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 msgr2=0x7fea60158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:02.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea60158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 -- 172.21.15.144:0/2988547660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009a70 con 0x7fea60071040 2023-11-28T11:51:02.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea5dd9b700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 0x7fea6014eef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea50009130 tx=0x7fea50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:02.659 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.647+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50004450 con 0x7fea60071040 2023-11-28T11:51:02.659 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5001e070 con 0x7fea60071040 2023-11-28T11:51:02.659 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea64b4d700 1 -- 172.21.15.144:0/2988547660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea50009550 con 0x7fea60071040 2023-11-28T11:51:02.659 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea500038e0 con 0x7fea60071040 2023-11-28T11:51:02.659 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea64b4d700 1 -- 172.21.15.144:0/2988547660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6015ccd0 con 0x7fea60071040 2023-11-28T11:51:02.660 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fea5000f050 con 0x7fea60071040 2023-11-28T11:51:02.660 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fea5001ad70 con 0x7fea60071040 2023-11-28T11:51:02.660 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.648+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea380052f0 con 0x7fea60071040 2023-11-28T11:51:02.660 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.651+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fea50032060 con 0x7fea60071040 2023-11-28T11:51:02.711 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:02.709+0000 7fa453583200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:51:02.722 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.721+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea38005870 con 0x7fea60071040 2023-11-28T11:51:02.723 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.721+0000 7fea4effd700 1 -- 172.21.15.144:0/2988547660 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fea50020070 con 0x7fea60071040 2023-11-28T11:51:02.723 INFO:teuthology.misc.health.smithi144.stdout: 2023-11-28T11:51:02.723 INFO:teuthology.misc.health.smithi144.stdout:{"epoch":10,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:01.426199+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:02.724 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 msgr2=0x7fea6014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:02.724 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 0x7fea6014eef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea50009130 tx=0x7fea50004750 comp rx=0 tx=0).stop 2023-11-28T11:51:02.724 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 shutdown_connections 2023-11-28T11:51:02.724 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea60160540 0x7fea6014f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.724 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6006dee0 0x7fea60158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.724 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 --2- 172.21.15.144:0/2988547660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea60071040 0x7fea6014eef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:02.725 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 >> 172.21.15.144:0/2988547660 conn(0x7fea600d0000 msgr2=0x7fea6015e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:02.725 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 shutdown_connections 2023-11-28T11:51:02.725 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:02.722+0000 7fea4cff9700 1 -- 172.21.15.144:0/2988547660 wait complete. 2023-11-28T11:51:02.735 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:51:02.771 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:02.769+0000 7f6e7a3cf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:51:02.845 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:02.844+0000 7fa453583200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:51:02.901 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:02.900+0000 7f6e7a3cf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:51:03.061 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:03.060+0000 7fa453583200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:51:03.118 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:03.117+0000 7f6e7a3cf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:51:03.218 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:03.216+0000 7fa453583200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:51:03.273 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:03.271+0000 7f6e7a3cf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:51:03.333 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:03.332+0000 7fa453583200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:51:03.386 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:03.384+0000 7f6e7a3cf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:51:03.656 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:03.654+0000 7fa453583200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:51:03.734 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:03.705+0000 7f6e7a3cf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:51:03.755 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:03.753+0000 7fa453583200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:51:03.807 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:03.806+0000 7f6e7a3cf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:51:03.939 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:03.937+0000 7fa453583200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:51:03.989 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:03.987+0000 7f6e7a3cf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:51:04.461 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:04.460+0000 7fa453583200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:51:04.510 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-11-28T11:51:04.508+0000 7f6e7a3cf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:51:05.366 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T11:51:05.365+0000 7f0405cc2640 -1 osd.6 0 log_to_monitors true 2023-11-28T11:51:05.422 INFO:tasks.ceph.osd.2.smithi144.stderr:2023-11-28T11:51:05.421+0000 7fa9e8740640 -1 osd.2 0 log_to_monitors true 2023-11-28T11:51:05.484 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:05.482+0000 7fa40a7b1700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T11:51:05.486 INFO:tasks.ceph.mgr.y.smithi144.stderr:2023-11-28T11:51:05.485+0000 7fa40a7b1700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T11:51:05.608 INFO:tasks.ceph.osd.1.smithi144.stderr:2023-11-28T11:51:05.607+0000 7feae6d0f640 -1 osd.1 0 log_to_monitors true 2023-11-28T11:51:05.640 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T11:51:05.639+0000 7f7d199a7640 -1 osd.5 0 log_to_monitors true 2023-11-28T11:51:05.640 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T11:51:05.639+0000 7f4845171640 -1 osd.7 0 log_to_monitors true 2023-11-28T11:51:05.647 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T11:51:05.646+0000 7fb21a6ac640 -1 osd.4 0 log_to_monitors true 2023-11-28T11:51:05.688 INFO:tasks.ceph.osd.3.smithi144.stderr:2023-11-28T11:51:05.686+0000 7fdcb2c9c640 -1 osd.3 0 log_to_monitors true 2023-11-28T11:51:05.694 INFO:tasks.ceph.osd.0.smithi144.stderr:2023-11-28T11:51:05.693+0000 7f0da9b6e640 -1 osd.0 0 log_to_monitors true 2023-11-28T11:51:08.424 INFO:tasks.ceph.osd.3.smithi144.stderr:2023-11-28T11:51:08.420+0000 7fdca6853700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T11:51:08.435 INFO:tasks.ceph.osd.3.smithi144.stderr:2023-11-28T11:51:08.432+0000 7fdc9ec2e700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.475 INFO:tasks.ceph.osd.1.smithi144.stderr:2023-11-28T11:51:08.474+0000 7feada8c6700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T11:51:08.488 INFO:tasks.ceph.osd.2.smithi144.stderr:2023-11-28T11:51:08.486+0000 7fa9ddafa700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T11:51:08.502 INFO:tasks.ceph.osd.1.smithi144.stderr:2023-11-28T11:51:08.500+0000 7fead2ca1700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.502 INFO:tasks.ceph.osd.2.smithi144.stderr:2023-11-28T11:51:08.500+0000 7fa9d46d2700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.654 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T11:51:08.654+0000 7f7d0d55e700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T11:51:08.680 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-11-28T11:51:08.680+0000 7f7d05939700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.704 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T11:51:08.703+0000 7fb20fa66700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T11:51:08.710 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T11:51:08.710+0000 7f4838d28700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T11:51:08.721 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-11-28T11:51:08.719+0000 7fb20663e700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.726 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T11:51:08.726+0000 7f03f9879700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T11:51:08.727 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-11-28T11:51:08.727+0000 7f4831103700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.741 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-11-28T11:51:08.741+0000 7f03f1c54700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:08.988 INFO:tasks.ceph.osd.0.smithi144.stderr:2023-11-28T11:51:08.987+0000 7f0d9d725700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T11:51:09.000 INFO:tasks.ceph.osd.0.smithi144.stderr:2023-11-28T11:51:08.999+0000 7f0d95b00700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:51:09.541 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:09.652 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.650+0000 7f09f0a8e700 1 Processor -- start 2023-11-28T11:51:09.652 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.650+0000 7f09f0a8e700 1 -- start start 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.651+0000 7f09f0a8e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 0x7f09ec140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.651+0000 7f09f0a8e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f09ec06f850 0x7f09ec06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.651+0000 7f09f0a8e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec13ff00 con 0x7f09ec0cdc90 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.651+0000 7f09f0a8e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec1404c0 con 0x7f09ec06f850 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.651+0000 7f09f0a8e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec15aa80 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 0x7f09ec140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ead9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f09ec06f850 0x7f09ec06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.655 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 0x7f09ec140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46088/0 (socket says 172.21.15.144:46088) 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 -- 172.21.15.144:0/1497979175 learned_addr learned my addr 172.21.15.144:0/1497979175 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09e9d9b700 1 --1- 172.21.15.144:0/1497979175 >> v1:172.21.15.177:6789/0 conn(0x7f09ec0cdc90 0x7f09ec0b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51208/0 (socket says 172.21.15.144:51208) 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 -- 172.21.15.144:0/1497979175 >> v2:172.21.15.177:3300/0 conn(0x7f09ec06f850 msgr2=0x7f09ec06fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 --2- 172.21.15.144:0/1497979175 >> v2:172.21.15.177:3300/0 conn(0x7f09ec06f850 0x7f09ec06fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 -- 172.21.15.144:0/1497979175 >> v1:172.21.15.177:6789/0 conn(0x7f09ec0cdc90 legacy=0x7f09ec0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 -- 172.21.15.144:0/1497979175 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0b3040 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ead9d700 1 --2- 172.21.15.144:0/1497979175 >> v2:172.21.15.177:3300/0 conn(0x7f09ec06f850 0x7f09ec06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09ea59c700 1 --2- 172.21.15.144:0/1497979175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 0x7f09ec140950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006fd0 tx=0x7f09e0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d55ce6d336fc5efc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09e959a700 1 -- 172.21.15.144:0/1497979175 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0005e30 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09e959a700 1 -- 172.21.15.144:0/1497979175 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e0004030 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.656 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.652+0000 7f09e959a700 1 -- 172.21.15.144:0/1497979175 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00043e0 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 -- 172.21.15.144:0/1497979175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 msgr2=0x7f09ec140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/1497979175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 0x7f09ec140950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006fd0 tx=0x7f09e0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 -- 172.21.15.144:0/1497979175 shutdown_connections 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/1497979175 >> v2:172.21.15.177:3300/0 conn(0x7f09ec06f850 0x7f09ec06fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/1497979175 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cd8c0 0x7f09ec140950 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 -- 172.21.15.144:0/1497979175 >> 172.21.15.144:0/1497979175 conn(0x7f09ec06aed0 msgr2=0x7f09ec06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:09.657 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 -- 172.21.15.144:0/1497979175 shutdown_connections 2023-11-28T11:51:09.658 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.653+0000 7f09f0a8e700 1 -- 172.21.15.144:0/1497979175 wait complete. 2023-11-28T11:51:09.661 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.654+0000 7f09f0a8e700 1 Processor -- start 2023-11-28T11:51:09.661 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.654+0000 7f09f0a8e700 1 -- start start 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09f0a8e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09ec06f850 0x7f09ec1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09f0a8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 0x7f09ec06bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09f0a8e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cdc90 0x7f09ec06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09f0a8e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec06c770 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09f0a8e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec1412b0 con 0x7f09ec06f850 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09f0a8e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0b3ea0 con 0x7f09ec0cdc90 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 0x7f09ec06bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 0x7f09ec06bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34014/0 (socket says 172.21.15.144:34014) 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 -- 172.21.15.144:0/787862717 learned_addr learned my addr 172.21.15.144:0/787862717 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09ead9d700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cdc90 0x7f09ec06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.662 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09ea59c700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09ec06f850 0x7f09ec1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 -- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cdc90 msgr2=0x7f09ec06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cdc90 0x7f09ec06cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 -- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09ec06f850 msgr2=0x7f09ec1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09ec06f850 0x7f09ec1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.655+0000 7f09e9d9b700 1 -- 172.21.15.144:0/787862717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4009a70 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.656+0000 7f09e9d9b700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 0x7f09ec06bd90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c3c0 tx=0x7f09dc00c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.656+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc005cb0 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.656+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc005e10 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.656+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e0009500 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.656+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc0072f0 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.656+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec0b35f0 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.663 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.657+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec0b3310 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.664 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.657+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f09dc019030 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.664 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.658+0000 7f09daffd700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f09cc0682a0 0x7f09cc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.664 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.658+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f09dc084910 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.664 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.658+0000 7f09ea59c700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f09cc0682a0 0x7f09cc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.664 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.658+0000 7f09ea59c700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f09cc0682a0 0x7f09cc06a750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006fd0 tx=0x7f09e0008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:09.664 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.662+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f09dc051760 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.851 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.849+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09ec140150 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.851 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.849+0000 7f09daffd700 1 -- 172.21.15.144:0/787862717 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10584 (secure 0 0 0) 0x7f09dc017070 con 0x7f09ec0cd8c0 2023-11-28T11:51:09.852 INFO:teuthology.misc.health.smithi144.stdout: 2023-11-28T11:51:09.852 INFO:teuthology.misc.health.smithi144.stdout:{"epoch":14,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:09.423704+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.851+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f09cc0682a0 msgr2=0x7f09cc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.851+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f09cc0682a0 0x7f09cc06a750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09e0006fd0 tx=0x7f09e0008070 comp rx=0 tx=0).stop 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.851+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 msgr2=0x7f09ec06bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.851+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 0x7f09ec06bd90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c3c0 tx=0x7f09dc00c780 comp rx=0 tx=0).stop 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 shutdown_connections 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f09cc0682a0 0x7f09cc06a750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.854 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f09ec0cdc90 0x7f09ec06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.855 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09ec06f850 0x7f09ec1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.855 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 --2- 172.21.15.144:0/787862717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09ec0cd8c0 0x7f09ec06bd90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.855 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 >> 172.21.15.144:0/787862717 conn(0x7f09ec06aed0 msgr2=0x7f09ec0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:09.855 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 shutdown_connections 2023-11-28T11:51:09.855 INFO:teuthology.misc.health.smithi144.stderr:2023-11-28T11:51:09.852+0000 7f09f0a8e700 1 -- 172.21.15.144:0/787862717 wait complete. 2023-11-28T11:51:09.867 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T11:51:09.867 INFO:tasks.ceph:Creating RBD pool 2023-11-28T11:51:09.867 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T11:51:09.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.992+0000 7f7aef1bd700 1 Processor -- start 2023-11-28T11:51:09.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.992+0000 7f7aef1bd700 1 -- start start 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aef1bd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae806dee0 0x7f7ae806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aef1bd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ae8160540 0x7f7ae81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aef1bd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae815d060 con 0x7f7ae8071040 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aef1bd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae815d340 con 0x7f7ae8160540 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aef1bd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae815d620 con 0x7f7ae806dee0 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aed75a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ae8160540 0x7f7ae81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aecf59700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7ae8071040 0x7f7ae8071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50780/0 (socket says 172.21.15.144:50780) 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aecf59700 1 -- 172.21.15.144:0/697461832 learned_addr learned my addr 172.21.15.144:0/697461832 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7ae7fff700 1 --2- 172.21.15.144:0/697461832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae806dee0 0x7f7ae806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:09.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aed75a700 1 -- 172.21.15.144:0/697461832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae806dee0 msgr2=0x7f7ae806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:09.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aed75a700 1 --2- 172.21.15.144:0/697461832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae806dee0 0x7f7ae806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:09.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aed75a700 1 -- 172.21.15.144:0/697461832 >> v1:172.21.15.144:6789/0 conn(0x7f7ae8071040 legacy=0x7f7ae8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:09.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aed75a700 1 -- 172.21.15.144:0/697461832 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae815d900 con 0x7f7ae8160540 2023-11-28T11:51:09.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.993+0000 7f7aed75a700 1 --2- 172.21.15.144:0/697461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae8160540 0x7f7ae81609c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009d90 tx=0x7f7ae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df55eb47863f5064 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.994+0000 7f7ae77fe700 1 -- 172.21.15.144:0/697461832 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000e070 con 0x7f7ae8160540 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.994+0000 7f7ae77fe700 1 -- 172.21.15.144:0/697461832 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae000bb20 con 0x7f7ae8160540 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.994+0000 7f7ae77fe700 1 -- 172.21.15.144:0/697461832 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0012430 con 0x7f7ae8160540 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 -- 172.21.15.144:0/697461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae8160540 msgr2=0x7f7ae81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/697461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae8160540 0x7f7ae81609c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009d90 tx=0x7f7ae000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 -- 172.21.15.144:0/697461832 shutdown_connections 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/697461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae8160540 0x7f7ae81609c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/697461832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae806dee0 0x7f7ae806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 -- 172.21.15.144:0/697461832 >> 172.21.15.144:0/697461832 conn(0x7f7ae80d0000 msgr2=0x7f7ae80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 -- 172.21.15.144:0/697461832 shutdown_connections 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.995+0000 7f7aef1bd700 1 -- 172.21.15.144:0/697461832 wait complete. 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.996+0000 7f7aef1bd700 1 Processor -- start 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.996+0000 7f7aef1bd700 1 -- start start 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aef1bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 0x7f7ae80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aef1bd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae8071040 0x7f7ae80c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aef1bd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae8160540 0x7f7ae80c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aef1bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81f62d0 con 0x7f7ae806dee0 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aef1bd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81f65b0 con 0x7f7ae8160540 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aef1bd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81f6890 con 0x7f7ae8071040 2023-11-28T11:51:10.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 0x7f7ae80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 0x7f7ae80b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34074/0 (socket says 172.21.15.144:34074) 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 -- 172.21.15.144:0/996924079 learned_addr learned my addr 172.21.15.144:0/996924079 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7ae7fff700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae8071040 0x7f7ae80c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aed75a700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae8160540 0x7f7ae80c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 -- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae8071040 msgr2=0x7f7ae80c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae8071040 0x7f7ae80c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 -- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae8160540 msgr2=0x7f7ae80c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae8160540 0x7f7ae80c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 -- 172.21.15.144:0/996924079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009a70 con 0x7f7ae806dee0 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7aecf59700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 0x7f7ae80b3990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c480 tx=0x7f7adc00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.997+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc005d70 con 0x7f7ae806dee0 2023-11-28T11:51:10.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.998+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc00ce90 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.998+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae815ccd0 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.998+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0072d0 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.998+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae806e870 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.999+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae806a270 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.999+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f7adc019030 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.999+0000 7f7ae57fa700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7acc0682a0 0x7f7acc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.999+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7adc084680 con 0x7f7ae806dee0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:09.999+0000 7f7ae7fff700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7acc0682a0 0x7f7acc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.000+0000 7f7ae7fff700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7acc0682a0 0x7f7acc06a750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ae815eb80 tx=0x7f7ad4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:10.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.004+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7adc04b5e0 con 0x7f7ae806dee0 2023-11-28T11:51:10.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.186+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f7ae80720a0 con 0x7f7ae806dee0 2023-11-28T11:51:10.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.432+0000 7f7ae57fa700 1 -- 172.21.15.144:0/996924079 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f7adc0170a0 con 0x7f7ae806dee0 2023-11-28T11:51:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7acc0682a0 msgr2=0x7f7acc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7acc0682a0 0x7f7acc06a750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ae815eb80 tx=0x7f7ad4009210 comp rx=0 tx=0).stop 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 msgr2=0x7f7ae80b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 0x7f7ae80b3990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c480 tx=0x7f7adc00c790 comp rx=0 tx=0).stop 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 shutdown_connections 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ae806dee0 0x7f7ae80b3990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7acc0682a0 0x7f7acc06a750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae8160540 0x7f7ae80c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 --2- 172.21.15.144:0/996924079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ae8071040 0x7f7ae80c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 >> 172.21.15.144:0/996924079 conn(0x7f7ae80d0000 msgr2=0x7f7ae815f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 shutdown_connections 2023-11-28T11:51:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.434+0000 7f7aef1bd700 1 -- 172.21.15.144:0/996924079 wait complete. 2023-11-28T11:51:10.438 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'rbd' created 2023-11-28T11:51:10.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T11:51:10.573 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.571+0000 7fb1df72e700 1 Processor -- start 2023-11-28T11:51:10.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.572+0000 7fb1df72e700 1 -- start start 2023-11-28T11:51:10.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.572+0000 7fb1df72e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d815cd50 0x7fb1d80d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.572+0000 7fb1df72e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb1d80b4ff0 0x7fb1d80cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.572+0000 7fb1df72e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d80cd380 con 0x7fb1d80b4470 2023-11-28T11:51:10.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.572+0000 7fb1df72e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d8066630 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1df72e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d8066070 con 0x7fb1d815cd50 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1ddccb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb1d80b4ff0 0x7fb1d80cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1dccc9700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb1d80b4470 0x7fb1d80b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50802/0 (socket says 172.21.15.144:50802) 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1dccc9700 1 -- 172.21.15.144:0/705218337 learned_addr learned my addr 172.21.15.144:0/705218337 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1dd4ca700 1 --2- 172.21.15.144:0/705218337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d815cd50 0x7fb1d80d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1ddccb700 1 -- 172.21.15.144:0/705218337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d815cd50 msgr2=0x7fb1d80d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1ddccb700 1 --2- 172.21.15.144:0/705218337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d815cd50 0x7fb1d80d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1ddccb700 1 -- 172.21.15.144:0/705218337 >> v1:172.21.15.144:6789/0 conn(0x7fb1d80b4470 legacy=0x7fb1d80b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1ddccb700 1 -- 172.21.15.144:0/705218337 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8066350 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1cffff700 1 -- 172.21.15.144:0/705218337 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909718095 0 0) 0x7fb1d80cd380 con 0x7fb1d80b4470 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1ddccb700 1 --2- 172.21.15.144:0/705218337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d80cdb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009d90 tx=0x7fb1d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d419220e2bc7662 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:10.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1cffff700 1 -- 172.21.15.144:0/705218337 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000e070 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1cffff700 1 -- 172.21.15.144:0/705218337 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d000bbd0 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.573+0000 7fb1cffff700 1 -- 172.21.15.144:0/705218337 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0012430 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 -- 172.21.15.144:0/705218337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 msgr2=0x7fb1d80cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 --2- 172.21.15.144:0/705218337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d80cdb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009d90 tx=0x7fb1d000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 -- 172.21.15.144:0/705218337 shutdown_connections 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 --2- 172.21.15.144:0/705218337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d80cdb40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 --2- 172.21.15.144:0/705218337 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d815cd50 0x7fb1d80d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 -- 172.21.15.144:0/705218337 >> 172.21.15.144:0/705218337 conn(0x7fb1d80b3080 msgr2=0x7fb1d80cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 -- 172.21.15.144:0/705218337 shutdown_connections 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.574+0000 7fb1df72e700 1 -- 172.21.15.144:0/705218337 wait complete. 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.575+0000 7fb1df72e700 1 Processor -- start 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.575+0000 7fb1df72e700 1 -- start start 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d80b4470 0x7fb1d81f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d809cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d815cd50 0x7fb1d80af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d806d130 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80b1880 con 0x7fb1d815cd50 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81f2a90 con 0x7fb1d80b4470 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d809cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d809cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34098/0 (socket says 172.21.15.144:34098) 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 -- 172.21.15.144:0/1540074005 learned_addr learned my addr 172.21.15.144:0/1540074005 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dd4ca700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d80b4470 0x7fb1d81f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1ddccb700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d815cd50 0x7fb1d80af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 -- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d80b4470 msgr2=0x7fb1d81f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d80b4470 0x7fb1d81f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 -- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d815cd50 msgr2=0x7fb1d80af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:10.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d815cd50 0x7fb1d80af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 -- 172.21.15.144:0/1540074005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0009a70 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1ddccb700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d815cd50 0x7fb1d80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1dccc9700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d809cb40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c580 tx=0x7fb1d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4004c80 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d4004e00 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.576+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d80ae8f0 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.577+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4007310 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.577+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d80cd620 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.578+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d80aea80 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.578+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fb1d401a020 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.578+0000 7fb1cdffb700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1bc0682a0 0x7fb1bc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.578+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4813+0+0 (secure 0 0 0) 0x7fb1d4085300 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.578+0000 7fb1dd4ca700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1bc0682a0 0x7fb1bc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:10.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.579+0000 7fb1dd4ca700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1bc0682a0 0x7fb1bc06a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009930 tx=0x7fb1c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:10.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.583+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1d4052030 con 0x7fb1d80b4ff0 2023-11-28T11:51:10.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:10.768+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb1d806d910 con 0x7fb1d80b4ff0 2023-11-28T11:51:11.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.443+0000 7fb1cdffb700 1 -- 172.21.15.144:0/1540074005 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb1d4055650 con 0x7fb1d80b4ff0 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1bc0682a0 msgr2=0x7fb1bc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1bc0682a0 0x7fb1bc06a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009930 tx=0x7fb1c8008040 comp rx=0 tx=0).stop 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 msgr2=0x7fb1d809cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d809cb40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c580 tx=0x7fb1d400c940 comp rx=0 tx=0).stop 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 shutdown_connections 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1bc0682a0 0x7fb1bc06a750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d815cd50 0x7fb1d80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1d80b4470 0x7fb1d81f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 --2- 172.21.15.144:0/1540074005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1d80b4ff0 0x7fb1d809cb40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 >> 172.21.15.144:0/1540074005 conn(0x7fb1d80b3080 msgr2=0x7fb1d80cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:11.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 shutdown_connections 2023-11-28T11:51:11.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.446+0000 7fb1df72e700 1 -- 172.21.15.144:0/1540074005 wait complete. 2023-11-28T11:51:11.449 INFO:teuthology.orchestra.run.smithi144.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T11:51:11.468 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T11:51:11.469 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:51:11.470 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:51:11.472 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:51:11.472 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:51:11.472 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:51:11.477 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:51:11.477 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:51:11.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:51:11.480 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:51:11.480 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:51:11.480 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:51:11.482 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:51:11.482 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T11:51:11.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:11.552 INFO:tasks.ceph.mds.c.smithi144.stdout:starting mds.c at 2023-11-28T11:51:11.553 INFO:tasks.ceph.mds.a.smithi144.stdout:starting mds.a at 2023-11-28T11:51:11.567 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T11:51:11.572 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T11:51:11.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.611+0000 7fc96c15e700 1 Processor -- start 2023-11-28T11:51:11.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.612+0000 7fc96c15e700 1 -- start start 2023-11-28T11:51:11.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.612+0000 7fc96c15e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc96406c990 0x7fc96406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.612+0000 7fc96c15e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9640ca240 0x7fc9640ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.612+0000 7fc96c15e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96406ae60 con 0x7fc96406e9f0 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.612+0000 7fc96c15e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96406d350 con 0x7fc96406c990 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.612+0000 7fc96c15e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9640cab70 con 0x7fc9640ca240 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc96406c990 0x7fc96406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc96406c990 0x7fc96406ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34138/0 (socket says 172.21.15.144:34138) 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 -- 172.21.15.144:0/3751804058 learned_addr learned my addr 172.21.15.144:0/3751804058 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc96a6fb700 1 --2- 172.21.15.144:0/3751804058 >> v2:172.21.15.177:3300/0 conn(0x7fc9640ca240 0x7fc9640ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 -- 172.21.15.144:0/3751804058 >> v2:172.21.15.177:3300/0 conn(0x7fc9640ca240 msgr2=0x7fc9640ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 --2- 172.21.15.144:0/3751804058 >> v2:172.21.15.177:3300/0 conn(0x7fc9640ca240 0x7fc9640ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 -- 172.21.15.144:0/3751804058 >> v1:172.21.15.177:6789/0 conn(0x7fc96406e9f0 legacy=0x7fc96406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 -- 172.21.15.144:0/3751804058 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96406a270 con 0x7fc96406c990 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc9696f9700 1 --2- 172.21.15.144:0/3751804058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96406c990 0x7fc96406ce10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc964070490 tx=0x7fc95800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42953a5d1c01c0c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc968ef8700 1 -- 172.21.15.144:0/3751804058 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800dc20 con 0x7fc96406c990 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc968ef8700 1 -- 172.21.15.144:0/3751804058 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800dd80 con 0x7fc96406c990 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.613+0000 7fc968ef8700 1 -- 172.21.15.144:0/3751804058 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958017630 con 0x7fc96406c990 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 -- 172.21.15.144:0/3751804058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96406c990 msgr2=0x7fc96406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3751804058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96406c990 0x7fc96406ce10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc964070490 tx=0x7fc95800be70 comp rx=0 tx=0).stop 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 -- 172.21.15.144:0/3751804058 shutdown_connections 2023-11-28T11:51:11.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3751804058 >> v2:172.21.15.177:3300/0 conn(0x7fc9640ca240 0x7fc9640ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3751804058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc96406c990 0x7fc96406ce10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 -- 172.21.15.144:0/3751804058 >> 172.21.15.144:0/3751804058 conn(0x7fc9640b12b0 msgr2=0x7fc9640b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:11.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 -- 172.21.15.144:0/3751804058 shutdown_connections 2023-11-28T11:51:11.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.614+0000 7fc96c15e700 1 -- 172.21.15.144:0/3751804058 wait complete. 2023-11-28T11:51:11.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.615+0000 7fc96c15e700 1 Processor -- start 2023-11-28T11:51:11.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.615+0000 7fc96c15e700 1 -- start start 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc96406c990 0x7fc9640addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc96406e9f0 0x7fc96415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 0x7fc96414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96414ae80 con 0x7fc9640ca240 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640ad940 con 0x7fc96406c990 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640acd30 con 0x7fc96406e9f0 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 0x7fc96414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 0x7fc96414b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34146/0 (socket says 172.21.15.144:34146) 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 -- 172.21.15.144:0/3321110281 learned_addr learned my addr 172.21.15.144:0/3321110281 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc969efa700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc96406c990 0x7fc9640addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc9696f9700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc96406e9f0 0x7fc96415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 -- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc96406e9f0 msgr2=0x7fc96415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc96406e9f0 0x7fc96415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 -- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc96406c990 msgr2=0x7fc9640addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc96406c990 0x7fc9640addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 -- 172.21.15.144:0/3321110281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95800b7f0 con 0x7fc9640ca240 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96a6fb700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 0x7fc96414b0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00e810 tx=0x7fc95c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00c070 con 0x7fc9640ca240 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c013610 con 0x7fc9640ca240 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96406a810 con 0x7fc9640ca240 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0139c0 con 0x7fc9640ca240 2023-11-28T11:51:11.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.616+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96406a530 con 0x7fc9640ca240 2023-11-28T11:51:11.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.617+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96406aaf0 con 0x7fc9640ca240 2023-11-28T11:51:11.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.618+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fc95c023070 con 0x7fc9640ca240 2023-11-28T11:51:11.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.618+0000 7fc9627fc700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9500682a0 0x7fc95006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.618+0000 7fc969efa700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9500682a0 0x7fc95006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.618+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fc95c085590 con 0x7fc9640ca240 2023-11-28T11:51:11.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.618+0000 7fc969efa700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9500682a0 0x7fc95006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc9540012e0 tx=0x7fc954009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:11.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.622+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc95c0521a0 con 0x7fc9640ca240 2023-11-28T11:51:11.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.808+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc96414be00 con 0x7fc9640ca240 2023-11-28T11:51:11.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.809+0000 7fc9627fc700 1 -- 172.21.15.144:0/3321110281 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14273 (secure 0 0 0) 0x7fc96414be00 con 0x7fc9640ca240 2023-11-28T11:51:11.810 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:11.810 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:11.428933+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:11.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9500682a0 msgr2=0x7fc95006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9500682a0 0x7fc95006a750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc9540012e0 tx=0x7fc954009040 comp rx=0 tx=0).stop 2023-11-28T11:51:11.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 msgr2=0x7fc96414b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 0x7fc96414b0f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00e810 tx=0x7fc95c00eb20 comp rx=0 tx=0).stop 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 shutdown_connections 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9640ca240 0x7fc96414b0f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9500682a0 0x7fc95006a750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc96406c990 0x7fc9640addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 --2- 172.21.15.144:0/3321110281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc96406e9f0 0x7fc96415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 >> 172.21.15.144:0/3321110281 conn(0x7fc9640b12b0 msgr2=0x7fc9640ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.811+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 shutdown_connections 2023-11-28T11:51:11.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.812+0000 7fc96c15e700 1 -- 172.21.15.144:0/3321110281 wait complete. 2023-11-28T11:51:11.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:11.825 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:11.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.947+0000 7f1fcbeae700 1 Processor -- start 2023-11-28T11:51:11.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.947+0000 7f1fcbeae700 1 -- start start 2023-11-28T11:51:11.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fcbeae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fcbeae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1fc406dee0 0x7f1fc406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fcbeae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1fc406cdc0 0x7f1fc406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fcbeae700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc4071950 con 0x7f1fc406cdc0 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fcbeae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc415d060 con 0x7f1fc406dee0 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fcbeae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc415d340 con 0x7f1fc4071040 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fca44b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1fc406cdc0 0x7f1fc406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fca44b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1fc406cdc0 0x7f1fc406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34158/0 (socket says 172.21.15.144:34158) 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fca44b700 1 -- 172.21.15.144:0/3901285776 learned_addr learned my addr 172.21.15.144:0/3901285776 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fc9c4a700 1 --2- 172.21.15.144:0/3901285776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.948+0000 7f1fc9449700 1 --2- 172.21.15.144:0/3901285776 >> v2:172.21.15.177:3300/0 conn(0x7f1fc406dee0 0x7f1fc406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fca44b700 1 -- 172.21.15.144:0/3901285776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 msgr2=0x7f1fc4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:11.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fca44b700 1 --2- 172.21.15.144:0/3901285776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fca44b700 1 -- 172.21.15.144:0/3901285776 >> v2:172.21.15.177:3300/0 conn(0x7f1fc406dee0 msgr2=0x7f1fc406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fca44b700 1 --2- 172.21.15.144:0/3901285776 >> v2:172.21.15.177:3300/0 conn(0x7f1fc406dee0 0x7f1fc406e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fca44b700 1 -- 172.21.15.144:0/3901285776 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc415d620 con 0x7f1fc406cdc0 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fca44b700 1 --2- 172.21.15.144:0/3901285776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406cdc0 0x7f1fc406d240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fc409e140 tx=0x7f1fbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f237bb50891bfb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fc8c48700 1 -- 172.21.15.144:0/3901285776 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00e070 con 0x7f1fc406cdc0 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fc8c48700 1 -- 172.21.15.144:0/3901285776 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc00b910 con 0x7f1fc406cdc0 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fc8c48700 1 -- 172.21.15.144:0/3901285776 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00bcc0 con 0x7f1fc406cdc0 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fcbeae700 1 -- 172.21.15.144:0/3901285776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406cdc0 msgr2=0x7f1fc406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.949+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/3901285776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406cdc0 0x7f1fc406d240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fc409e140 tx=0x7f1fbc00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 -- 172.21.15.144:0/3901285776 shutdown_connections 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/3901285776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406cdc0 0x7f1fc406d240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/3901285776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/3901285776 >> v2:172.21.15.177:3300/0 conn(0x7f1fc406dee0 0x7f1fc406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 -- 172.21.15.144:0/3901285776 >> 172.21.15.144:0/3901285776 conn(0x7f1fc40d0000 msgr2=0x7f1fc40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:11.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 -- 172.21.15.144:0/3901285776 shutdown_connections 2023-11-28T11:51:11.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.950+0000 7f1fcbeae700 1 -- 172.21.15.144:0/3901285776 wait complete. 2023-11-28T11:51:11.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.951+0000 7f1fcbeae700 1 Processor -- start 2023-11-28T11:51:11.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.951+0000 7f1fcbeae700 1 -- start start 2023-11-28T11:51:11.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.951+0000 7f1fcbeae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fc406cdc0 0x7f1fc4144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.951+0000 7f1fcbeae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 0x7f1fc40c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.951+0000 7f1fcbeae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc415bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.951+0000 7f1fcbeae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc41454e0 con 0x7f1fc406dee0 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fcbeae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40c64b0 con 0x7f1fc406cdc0 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fcbeae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40c5ae0 con 0x7f1fc4071040 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 0x7f1fc40c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 0x7f1fc40c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34162/0 (socket says 172.21.15.144:34162) 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 -- 172.21.15.144:0/940000166 learned_addr learned my addr 172.21.15.144:0/940000166 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fca44b700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc415bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9c4a700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fc406cdc0 0x7f1fc4144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 -- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 msgr2=0x7f1fc415bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc415bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 -- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fc406cdc0 msgr2=0x7f1fc4144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fc406cdc0 0x7f1fc4144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 -- 172.21.15.144:0/940000166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009b30 con 0x7f1fc406dee0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fc9449700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 0x7f1fc40c5f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000e8d0 tx=0x7f1fc000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0013680 con 0x7f1fc406dee0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc00137e0 con 0x7f1fc406dee0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc406e890 con 0x7f1fc406dee0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0013b90 con 0x7f1fc406dee0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.952+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc415d900 con 0x7f1fc406dee0 2023-11-28T11:51:11.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.953+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc415ccd0 con 0x7f1fc406dee0 2023-11-28T11:51:11.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.954+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f1fc0019030 con 0x7f1fc406dee0 2023-11-28T11:51:11.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.954+0000 7f1fba7fc700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fa80682a0 0x7f1fa806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:11.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.954+0000 7f1fc9c4a700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fa80682a0 0x7f1fa806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:11.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.954+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f1fc0085570 con 0x7f1fc406dee0 2023-11-28T11:51:11.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.955+0000 7f1fc9c4a700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fa80682a0 0x7f1fa806a750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009b40 tx=0x7f1fb4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:11.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:11.958+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1fc0052180 con 0x7f1fc406dee0 2023-11-28T11:51:12.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.139+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1fc40c5c90 con 0x7f1fc406dee0 2023-11-28T11:51:12.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.140+0000 7f1fba7fc700 1 -- 172.21.15.144:0/940000166 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1fc40c5c90 con 0x7f1fc406dee0 2023-11-28T11:51:12.141 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fa80682a0 msgr2=0x7f1fa806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fa80682a0 0x7f1fa806a750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009b40 tx=0x7f1fb4009210 comp rx=0 tx=0).stop 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 msgr2=0x7f1fc40c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 0x7f1fc40c5f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000e8d0 tx=0x7f1fc000ebe0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 shutdown_connections 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fa80682a0 0x7f1fa806a750 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc4071040 0x7f1fc415bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fc406cdc0 0x7f1fc4144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 --2- 172.21.15.144:0/940000166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc406dee0 0x7f1fc40c5f70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 >> 172.21.15.144:0/940000166 conn(0x7f1fc40d0000 msgr2=0x7f1fc415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:12.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 shutdown_connections 2023-11-28T11:51:12.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.142+0000 7f1fcbeae700 1 -- 172.21.15.144:0/940000166 wait complete. 2023-11-28T11:51:12.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:12.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.277+0000 7fbb24caa700 1 Processor -- start 2023-11-28T11:51:12.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.277+0000 7fbb24caa700 1 -- start start 2023-11-28T11:51:12.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb24caa700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb201658c0 0x7fbb20167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb24caa700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2015d060 con 0x7fbb20071040 2023-11-28T11:51:12.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb24caa700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2015d340 con 0x7fbb2006dee0 2023-11-28T11:51:12.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb24caa700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb20161550 con 0x7fbb201658c0 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1e59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbb20071040 0x7fbb20071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50880/0 (socket says 172.21.15.144:50880) 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1e59c700 1 -- 172.21.15.144:0/1321168406 learned_addr learned my addr 172.21.15.144:0/1321168406 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1ed9d700 1 --2- 172.21.15.144:0/1321168406 >> v2:172.21.15.177:3300/0 conn(0x7fbb201658c0 0x7fbb20167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380331699 0 0) 0x7fbb2015d060 con 0x7fbb20071040 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb080035b0 con 0x7fbb20071040 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436517229 0 0) 0x7fbb080035b0 con 0x7fbb20071040 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 >> v2:172.21.15.177:3300/0 conn(0x7fbb201658c0 msgr2=0x7fbb20167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 --2- 172.21.15.144:0/1321168406 >> v2:172.21.15.177:3300/0 conn(0x7fbb201658c0 0x7fbb20167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 >> v1:172.21.15.177:6789/0 conn(0x7fbb2006dee0 legacy=0x7fbb2006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2015d620 con 0x7fbb20071040 2023-11-28T11:51:12.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fbb10001670 con 0x7fbb20071040 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 >> v1:172.21.15.144:6789/0 conn(0x7fbb20071040 legacy=0x7fbb20071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb08003a00 0x7fbb08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb08004850 0x7fbb08004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.278+0000 7fbb1d59a700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08005320 0x7fbb0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20161550 con 0x7fbb08004850 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0800c250 con 0x7fbb08005320 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0800c410 con 0x7fbb08003a00 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb08004850 0x7fbb08004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1ed9d700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08005320 0x7fbb0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1e59c700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb08003a00 0x7fbb08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 -- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb08003a00 msgr2=0x7fbb08003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb08003a00 0x7fbb08003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 -- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08005320 msgr2=0x7fbb0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08005320 0x7fbb0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1ed9d700 1 -- 172.21.15.144:0/1321168406 reap_dead start 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 -- 172.21.15.144:0/1321168406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080035b0 con 0x7fbb08004850 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1dd9b700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb08004850 0x7fbb08004cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001800 tx=0x7fbb08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0f849e6d5bcc87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140012f0 con 0x7fbb08004850 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb14005310 con 0x7fbb08004850 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.279+0000 7fbb1d59a700 1 -- 172.21.15.144:0/1321168406 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14006860 con 0x7fbb08004850 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 -- 172.21.15.144:0/1321168406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb08004850 msgr2=0x7fbb08004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb08004850 0x7fbb08004cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001800 tx=0x7fbb08005880 comp rx=0 tx=0).stop 2023-11-28T11:51:12.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 -- 172.21.15.144:0/1321168406 shutdown_connections 2023-11-28T11:51:12.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1321168406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb08004850 0x7fbb08004cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 -- 172.21.15.144:0/1321168406 >> 172.21.15.144:0/1321168406 conn(0x7fbb200d0000 msgr2=0x7fbb200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:12.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 -- 172.21.15.144:0/1321168406 shutdown_connections 2023-11-28T11:51:12.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.280+0000 7fbb24caa700 1 -- 172.21.15.144:0/1321168406 wait complete. 2023-11-28T11:51:12.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.281+0000 7fbb24caa700 1 Processor -- start 2023-11-28T11:51:12.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 -- start start 2023-11-28T11:51:12.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08004850 0x7fbb20141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb200acdf0 0x7fbb200ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 0x7fbb20142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201571b0 con 0x7fbb20142850 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200ad360 con 0x7fbb08004850 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.282+0000 7fbb24caa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200ad640 con 0x7fbb200acdf0 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 0x7fbb20142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 0x7fbb20142cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34184/0 (socket says 172.21.15.144:34184) 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 -- 172.21.15.144:0/1622313595 learned_addr learned my addr 172.21.15.144:0/1622313595 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1dd9b700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb200acdf0 0x7fbb200ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1e59c700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08004850 0x7fbb20141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 -- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb200acdf0 msgr2=0x7fbb200ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb200acdf0 0x7fbb200ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 -- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08004850 msgr2=0x7fbb20141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08004850 0x7fbb20141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 -- 172.21.15.144:0/1622313595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2015ccd0 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb1ed9d700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 0x7fbb20142cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb20071d40 tx=0x7fbb18011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800e040 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1800f040 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800fa00 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2015d900 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.283+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2006a270 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.284+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb200720a0 con 0x7fbb20142850 2023-11-28T11:51:12.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.284+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fbb1800fb60 con 0x7fbb20142850 2023-11-28T11:51:12.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.285+0000 7fbb0effd700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682a0 0x7fbb0006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.285+0000 7fbb1e59c700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682a0 0x7fbb0006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.285+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fbb18086160 con 0x7fbb20142850 2023-11-28T11:51:12.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.285+0000 7fbb1e59c700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682a0 0x7fbb0006a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb200c6a60 tx=0x7fbb10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:12.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.289+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb18052d70 con 0x7fbb20142850 2023-11-28T11:51:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.474+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb20073420 con 0x7fbb20142850 2023-11-28T11:51:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.474+0000 7fbb0effd700 1 -- 172.21.15.144:0/1622313595 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb1800fdf0 con 0x7fbb20142850 2023-11-28T11:51:12.475 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682a0 msgr2=0x7fbb0006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682a0 0x7fbb0006a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb200c6a60 tx=0x7fbb10009040 comp rx=0 tx=0).stop 2023-11-28T11:51:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 msgr2=0x7fbb20142cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 0x7fbb20142cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb20071d40 tx=0x7fbb18011f80 comp rx=0 tx=0).stop 2023-11-28T11:51:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 shutdown_connections 2023-11-28T11:51:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb20142850 0x7fbb20142cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682a0 0x7fbb0006a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb08004850 0x7fbb20141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 --2- 172.21.15.144:0/1622313595 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb200acdf0 0x7fbb200ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 >> 172.21.15.144:0/1622313595 conn(0x7fbb200d0000 msgr2=0x7fbb2015e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:12.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 shutdown_connections 2023-11-28T11:51:12.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.477+0000 7fbb24caa700 1 -- 172.21.15.144:0/1622313595 wait complete. 2023-11-28T11:51:12.491 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:12.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 Processor -- start 2023-11-28T11:51:12.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 -- start start 2023-11-28T11:51:12.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa700cd8c0 0x7faa70140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa7006f850 0x7faa7006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7013ff00 con 0x7faa700cdc90 2023-11-28T11:51:12.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa701404c0 con 0x7faa700cd8c0 2023-11-28T11:51:12.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.619+0000 7faa76a1f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7015aa80 con 0x7faa7006f850 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa700cd8c0 0x7faa70140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7faa700cdc90 0x7faa700b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50924/0 (socket says 172.21.15.144:50924) 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa700cd8c0 0x7faa70140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34210/0 (socket says 172.21.15.144:34210) 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6f7fe700 1 -- 172.21.15.144:0/2060250718 learned_addr learned my addr 172.21.15.144:0/2060250718 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa74fbc700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa7006f850 0x7faa7006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 -- 172.21.15.144:0/2060250718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa7006f850 msgr2=0x7faa7006fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa7006f850 0x7faa7006fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 -- 172.21.15.144:0/2060250718 >> v1:172.21.15.144:6789/0 conn(0x7faa700cdc90 legacy=0x7faa700b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 -- 172.21.15.144:0/2060250718 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700b3040 con 0x7faa700cd8c0 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6effd700 1 -- 172.21.15.144:0/2060250718 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28304620 0 0) 0x7faa7013ff00 con 0x7faa700cdc90 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa74fbc700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa7006f850 0x7faa7006fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6ffff700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa700cd8c0 0x7faa70140950 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa60009820 tx=0x7faa60009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2933d5446f1e60dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:12.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6effd700 1 -- 172.21.15.144:0/2060250718 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60005e30 con 0x7faa700cd8c0 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6effd700 1 -- 172.21.15.144:0/2060250718 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa60006030 con 0x7faa700cd8c0 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.620+0000 7faa6effd700 1 -- 172.21.15.144:0/2060250718 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600063e0 con 0x7faa700cd8c0 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 -- 172.21.15.144:0/2060250718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa700cd8c0 msgr2=0x7faa70140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa700cd8c0 0x7faa70140950 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa60009820 tx=0x7faa60009b30 comp rx=0 tx=0).stop 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 -- 172.21.15.144:0/2060250718 shutdown_connections 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa7006f850 0x7faa7006fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2060250718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa700cd8c0 0x7faa70140950 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 -- 172.21.15.144:0/2060250718 >> 172.21.15.144:0/2060250718 conn(0x7faa7006aed0 msgr2=0x7faa7006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 -- 172.21.15.144:0/2060250718 shutdown_connections 2023-11-28T11:51:12.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.621+0000 7faa76a1f700 1 -- 172.21.15.144:0/2060250718 wait complete. 2023-11-28T11:51:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.622+0000 7faa76a1f700 1 Processor -- start 2023-11-28T11:51:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 -- start start 2023-11-28T11:51:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 0x7faa70143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa700cd8c0 0x7faa700ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa700cdc90 0x7faa701540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700caab0 con 0x7faa7006f850 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa701427d0 con 0x7faa700cdc90 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.623+0000 7faa76a1f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700b4b60 con 0x7faa700cd8c0 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 0x7faa70143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 0x7faa70143590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34216/0 (socket says 172.21.15.144:34216) 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 -- 172.21.15.144:0/2877120484 learned_addr learned my addr 172.21.15.144:0/2877120484 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6f7fe700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa700cd8c0 0x7faa700ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa74fbc700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa700cdc90 0x7faa701540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 -- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa700cd8c0 msgr2=0x7faa700ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa700cd8c0 0x7faa700ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 -- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa700cdc90 msgr2=0x7faa701540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa700cdc90 0x7faa701540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 -- 172.21.15.144:0/2877120484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60009430 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa74fbc700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa700cdc90 0x7faa701540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6ffff700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 0x7faa70143590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa700b3aa0 tx=0x7faa60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60006950 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa60020070 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa700b35f0 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60025670 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.624+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa700b3310 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.625+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa70140150 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.626+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7faa6002b020 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.626+0000 7faa6cff9700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faa5c0682a0 0x7faa5c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.626+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faa600871c0 con 0x7faa7006f850 2023-11-28T11:51:12.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.626+0000 7faa6f7fe700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faa5c0682a0 0x7faa5c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:12.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.626+0000 7faa6f7fe700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faa5c0682a0 0x7faa5c06a750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa701574d0 tx=0x7faa64007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:12.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.630+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa60053d40 con 0x7faa7006f850 2023-11-28T11:51:12.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.871+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa701f44d0 con 0x7faa7006f850 2023-11-28T11:51:12.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.872+0000 7faa6cff9700 1 -- 172.21.15.144:0/2877120484 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7faa60057360 con 0x7faa7006f850 2023-11-28T11:51:12.873 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:12.873 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T11:51:12.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faa5c0682a0 msgr2=0x7faa5c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faa5c0682a0 0x7faa5c06a750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa701574d0 tx=0x7faa64007e70 comp rx=0 tx=0).stop 2023-11-28T11:51:12.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 msgr2=0x7faa70143590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:12.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 0x7faa70143590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa700b3aa0 tx=0x7faa60004750 comp rx=0 tx=0).stop 2023-11-28T11:51:12.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 shutdown_connections 2023-11-28T11:51:12.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7006f850 0x7faa70143590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faa5c0682a0 0x7faa5c06a750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa700cdc90 0x7faa701540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 --2- 172.21.15.144:0/2877120484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faa700cd8c0 0x7faa700ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 >> 172.21.15.144:0/2877120484 conn(0x7faa7006aed0 msgr2=0x7faa7013ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 shutdown_connections 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:12.874+0000 7faa76a1f700 1 -- 172.21.15.144:0/2877120484 wait complete. 2023-11-28T11:51:12.876 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 3 2023-11-28T11:51:12.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:13.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.009+0000 7f55e852e700 1 Processor -- start 2023-11-28T11:51:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.009+0000 7f55e852e700 1 -- start start 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e852e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f55e00cd8c0 0x7f55e0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e852e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e852e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e013ff00 con 0x7f55e00b2e70 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e852e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e01404c0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e852e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e015aa80 con 0x7f55e00cdc90 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e62ca700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f55e00cd8c0 0x7f55e0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e5ac9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e6acb700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f55e00b2e70 0x7f55e006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51440/0 (socket says 172.21.15.144:51440) 2023-11-28T11:51:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e6acb700 1 -- 172.21.15.144:0/1319752173 learned_addr learned my addr 172.21.15.144:0/1319752173 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e52c8700 1 -- 172.21.15.144:0/1319752173 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608247676 0 0) 0x7f55e013ff00 con 0x7f55e00b2e70 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e52c8700 1 -- 172.21.15.144:0/1319752173 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc0035b0 con 0x7f55e00b2e70 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e62ca700 1 -- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 msgr2=0x7f55e00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e62ca700 1 --2- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e62ca700 1 -- 172.21.15.144:0/1319752173 >> v1:172.21.15.177:6789/0 conn(0x7f55e00b2e70 legacy=0x7f55e006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e62ca700 1 -- 172.21.15.144:0/1319752173 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00b3310 con 0x7f55e00cd8c0 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e5ac9700 1 --2- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e62ca700 1 --2- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00cd8c0 0x7f55e0140950 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009820 tx=0x7f55dc009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd84786691ea525d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e52c8700 1 -- 172.21.15.144:0/1319752173 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc005e30 con 0x7f55e00cd8c0 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e52c8700 1 -- 172.21.15.144:0/1319752173 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc006030 con 0x7f55e00cd8c0 2023-11-28T11:51:13.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.010+0000 7f55e52c8700 1 -- 172.21.15.144:0/1319752173 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0063c0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 -- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00cd8c0 msgr2=0x7f55e0140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 --2- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00cd8c0 0x7f55e0140950 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009820 tx=0x7f55dc009b30 comp rx=0 tx=0).stop 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 -- 172.21.15.144:0/1319752173 shutdown_connections 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 --2- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00cd8c0 0x7f55e0140950 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 --2- 172.21.15.144:0/1319752173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 -- 172.21.15.144:0/1319752173 >> 172.21.15.144:0/1319752173 conn(0x7f55e006aed0 msgr2=0x7f55e006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 -- 172.21.15.144:0/1319752173 shutdown_connections 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.011+0000 7f55e852e700 1 -- 172.21.15.144:0/1319752173 wait complete. 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.012+0000 7f55e852e700 1 Processor -- start 2023-11-28T11:51:13.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.012+0000 7f55e852e700 1 -- start start 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e852e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 0x7f55e0159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e852e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 0x7f55e0142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e852e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e852e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e01430c0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e852e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00b3e30 con 0x7f55e00b2e70 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e852e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0142580 con 0x7f55e00cdc90 2023-11-28T11:51:13.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 0x7f55e0142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e62ca700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 0x7f55e0159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e6acb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 0x7f55e0142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34244/0 (socket says 172.21.15.144:34244) 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e62ca700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 0x7f55e0159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36514/0 (socket says 172.21.15.144:36514) 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 -- 172.21.15.144:0/159753557 learned_addr learned my addr 172.21.15.144:0/159753557 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 -- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 msgr2=0x7f55e015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 -- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 msgr2=0x7f55e0159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 0x7f55e0159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e5ac9700 1 -- 172.21.15.144:0/159753557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d00094a0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.013+0000 7f55e62ca700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 0x7f55e0159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.014+0000 7f55e5ac9700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 0x7f55e0142040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55d000ebb0 tx=0x7f55d000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.014+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000c850 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.014+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d000c9b0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.014+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc009430 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.014+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000cd40 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.014+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e0140150 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.015+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e00b35f0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.015+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f55d0019030 con 0x7f55e00cd8c0 2023-11-28T11:51:13.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.015+0000 7f55d6ffd700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55c40682a0 0x7f55c406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.015+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f55d00857b0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.015+0000 7f55e62ca700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55c40682a0 0x7f55c406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.016+0000 7f55e62ca700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55c40682a0 0x7f55c406a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000c00 tx=0x7f55dc008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.019+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55d00523b0 con 0x7f55e00cd8c0 2023-11-28T11:51:13.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.205+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55e00ce100 con 0x7f55e00cd8c0 2023-11-28T11:51:13.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.205+0000 7f55d6ffd700 1 -- 172.21.15.144:0/159753557 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14268 (secure 0 0 0) 0x7f55d0017090 con 0x7f55e00cd8c0 2023-11-28T11:51:13.206 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:13.206 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":17,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:12.439972+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:13.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.207+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55c40682a0 msgr2=0x7f55c406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.207+0000 7f55e852e700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55c40682a0 0x7f55c406a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000c00 tx=0x7f55dc008070 comp rx=0 tx=0).stop 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.207+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 msgr2=0x7f55e0142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.207+0000 7f55e852e700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 0x7f55e0142040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55d000ebb0 tx=0x7f55d000eec0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 shutdown_connections 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55c40682a0 0x7f55c406a750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55e00cdc90 0x7f55e015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e00b2e70 0x7f55e0159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 --2- 172.21.15.144:0/159753557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55e00cd8c0 0x7f55e0142040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 >> 172.21.15.144:0/159753557 conn(0x7f55e006aed0 msgr2=0x7f55e00d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 shutdown_connections 2023-11-28T11:51:13.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.208+0000 7f55e852e700 1 -- 172.21.15.144:0/159753557 wait complete. 2023-11-28T11:51:13.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:13.221 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:13.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.347+0000 7fa09f3a4700 1 Processor -- start 2023-11-28T11:51:13.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.347+0000 7fa09f3a4700 1 -- start start 2023-11-28T11:51:13.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09f3a4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa09806dee0 0x7fa09806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09f3a4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa098160540 0x7fa0981609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09f3a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09815d060 con 0x7fa098071040 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09f3a4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09815d340 con 0x7fa09806dee0 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09f3a4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09815d620 con 0x7fa098160540 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09c93f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa09806dee0 0x7fa09806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09c93f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa09806dee0 0x7fa09806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34258/0 (socket says 172.21.15.144:34258) 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09d140700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa098071040 0x7fa098071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50968/0 (socket says 172.21.15.144:50968) 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09d941700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa098160540 0x7fa0981609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09c93f700 1 -- 172.21.15.144:0/2131069745 learned_addr learned my addr 172.21.15.144:0/2131069745 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09c93f700 1 -- 172.21.15.144:0/2131069745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa098160540 msgr2=0x7fa0981609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09c93f700 1 --2- 172.21.15.144:0/2131069745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa098160540 0x7fa0981609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa09c93f700 1 -- 172.21.15.144:0/2131069745 >> v1:172.21.15.144:6789/0 conn(0x7fa098071040 legacy=0x7fa098071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.348+0000 7fa08ffff700 1 -- 172.21.15.144:0/2131069745 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711150886 0 0) 0x7fa09815d060 con 0x7fa098071040 2023-11-28T11:51:13.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09c93f700 1 -- 172.21.15.144:0/2131069745 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09815d900 con 0x7fa09806dee0 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09d941700 1 --2- 172.21.15.144:0/2131069745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa098160540 0x7fa0981609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09c93f700 1 --2- 172.21.15.144:0/2131069745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09806dee0 0x7fa09806e330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098f0 tx=0x7fa094009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36df5a1227e2f5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa08ffff700 1 -- 172.21.15.144:0/2131069745 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094011830 con 0x7fa09806dee0 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa08ffff700 1 -- 172.21.15.144:0/2131069745 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094011990 con 0x7fa09806dee0 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa08ffff700 1 -- 172.21.15.144:0/2131069745 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094011d20 con 0x7fa09806dee0 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 -- 172.21.15.144:0/2131069745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09806dee0 msgr2=0x7fa09806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/2131069745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09806dee0 0x7fa09806e330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098f0 tx=0x7fa094009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 -- 172.21.15.144:0/2131069745 shutdown_connections 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/2131069745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa098160540 0x7fa0981609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/2131069745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09806dee0 0x7fa09806e330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 -- 172.21.15.144:0/2131069745 >> 172.21.15.144:0/2131069745 conn(0x7fa0980d0000 msgr2=0x7fa0980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:13.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 -- 172.21.15.144:0/2131069745 shutdown_connections 2023-11-28T11:51:13.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.349+0000 7fa09f3a4700 1 -- 172.21.15.144:0/2131069745 wait complete. 2023-11-28T11:51:13.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.351+0000 7fa09f3a4700 1 Processor -- start 2023-11-28T11:51:13.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.351+0000 7fa09f3a4700 1 -- start start 2023-11-28T11:51:13.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09f3a4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa09806dee0 0x7fa098152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09f3a4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa098071040 0x7fa0980c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09f3a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 0x7fa0980b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09f3a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0980c43d0 con 0x7fa098160540 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09f3a4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098153390 con 0x7fa098071040 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09f3a4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0980c5f70 con 0x7fa09806dee0 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 0x7fa0980b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 0x7fa0980b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34266/0 (socket says 172.21.15.144:34266) 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 -- 172.21.15.144:0/1577883119 learned_addr learned my addr 172.21.15.144:0/1577883119 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d140700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa09806dee0 0x7fa098152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09c93f700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa098071040 0x7fa0980c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 -- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa09806dee0 msgr2=0x7fa098152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa09806dee0 0x7fa098152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 -- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa098071040 msgr2=0x7fa0980c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa098071040 0x7fa0980c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 -- 172.21.15.144:0/1577883119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa090009a70 con 0x7fa098160540 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.352+0000 7fa09d941700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 0x7fa0980b0280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa090009e60 tx=0x7fa09000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.353+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090009cd0 con 0x7fa098160540 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.353+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09000f040 con 0x7fa098160540 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.353+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000cd60 con 0x7fa098160540 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.353+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa094009550 con 0x7fa098160540 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.353+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09815ccd0 con 0x7fa098160540 2023-11-28T11:51:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.354+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09806e870 con 0x7fa098160540 2023-11-28T11:51:13.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.354+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fa090014480 con 0x7fa098160540 2023-11-28T11:51:13.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.355+0000 7fa08dffb700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa07c0682a0 0x7fa07c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.355+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa0900860c0 con 0x7fa098160540 2023-11-28T11:51:13.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.355+0000 7fa09d140700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa07c0682a0 0x7fa07c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.355+0000 7fa09d140700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa07c0682a0 0x7fa07c06a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa088000f60 tx=0x7fa088007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.359+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa090052bd0 con 0x7fa098160540 2023-11-28T11:51:13.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.545+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa09806a270 con 0x7fa098160540 2023-11-28T11:51:13.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.545+0000 7fa08dffb700 1 -- 172.21.15.144:0/1577883119 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa0900561f0 con 0x7fa098160540 2023-11-28T11:51:13.547 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:13.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa07c0682a0 msgr2=0x7fa07c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa07c0682a0 0x7fa07c06a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa088000f60 tx=0x7fa088007e70 comp rx=0 tx=0).stop 2023-11-28T11:51:13.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 msgr2=0x7fa0980b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 0x7fa0980b0280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa090009e60 tx=0x7fa09000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 shutdown_connections 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa098160540 0x7fa0980b0280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa07c0682a0 0x7fa07c06a750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa09806dee0 0x7fa098152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 --2- 172.21.15.144:0/1577883119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa098071040 0x7fa0980c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 >> 172.21.15.144:0/1577883119 conn(0x7fa0980d0000 msgr2=0x7fa0980cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 shutdown_connections 2023-11-28T11:51:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.547+0000 7fa09f3a4700 1 -- 172.21.15.144:0/1577883119 wait complete. 2023-11-28T11:51:13.560 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:13.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 Processor -- start 2023-11-28T11:51:13.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 -- start start 2023-11-28T11:51:13.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f35d8071040 0x7f35d8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 0x7f35d81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d8071950 con 0x7f35d806dee0 2023-11-28T11:51:13.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d815d060 con 0x7f35d8071040 2023-11-28T11:51:13.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.682+0000 7f35deaac700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d815d340 con 0x7f35d8160540 2023-11-28T11:51:13.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 0x7f35d81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dc848700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f35d8071040 0x7f35d8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 0x7f35d81609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46356/0 (socket says 172.21.15.144:46356) 2023-11-28T11:51:13.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 -- 172.21.15.144:0/1350559640 learned_addr learned my addr 172.21.15.144:0/1350559640 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:13.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 -- 172.21.15.144:0/1350559640 >> v2:172.21.15.177:3300/0 conn(0x7f35d8071040 msgr2=0x7f35d8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 --2- 172.21.15.144:0/1350559640 >> v2:172.21.15.177:3300/0 conn(0x7f35d8071040 0x7f35d8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 -- 172.21.15.144:0/1350559640 >> v1:172.21.15.177:6789/0 conn(0x7f35d806dee0 legacy=0x7f35d806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 -- 172.21.15.144:0/1350559640 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d815d620 con 0x7f35d8160540 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dc848700 1 --2- 172.21.15.144:0/1350559640 >> v2:172.21.15.177:3300/0 conn(0x7f35d8071040 0x7f35d8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35dd049700 1 --2- 172.21.15.144:0/1350559640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 0x7f35d81609c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009e50 tx=0x7f35d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e0c36e3efac81c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.683+0000 7f35d77fe700 1 -- 172.21.15.144:0/1350559640 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000e070 con 0x7f35d8160540 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35d77fe700 1 -- 172.21.15.144:0/1350559640 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000b910 con 0x7f35d8160540 2023-11-28T11:51:13.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35d77fe700 1 -- 172.21.15.144:0/1350559640 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000bcc0 con 0x7f35d8160540 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 -- 172.21.15.144:0/1350559640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 msgr2=0x7f35d81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 --2- 172.21.15.144:0/1350559640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 0x7f35d81609c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009e50 tx=0x7f35d000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 -- 172.21.15.144:0/1350559640 shutdown_connections 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 --2- 172.21.15.144:0/1350559640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8160540 0x7f35d81609c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 --2- 172.21.15.144:0/1350559640 >> v2:172.21.15.177:3300/0 conn(0x7f35d8071040 0x7f35d8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 -- 172.21.15.144:0/1350559640 >> 172.21.15.144:0/1350559640 conn(0x7f35d80d0000 msgr2=0x7f35d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 -- 172.21.15.144:0/1350559640 shutdown_connections 2023-11-28T11:51:13.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.684+0000 7f35deaac700 1 -- 172.21.15.144:0/1350559640 wait complete. 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.685+0000 7f35deaac700 1 Processor -- start 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.685+0000 7f35deaac700 1 -- start start 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35deaac700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35d806dee0 0x7f35d80c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35deaac700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8071040 0x7f35d814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35deaac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35d8160540 0x7f35d80b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35deaac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c50c0 con 0x7f35d8160540 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35deaac700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c5360 con 0x7f35d806dee0 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35deaac700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c5640 con 0x7f35d8071040 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35dc848700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35d806dee0 0x7f35d80c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35d7fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8071040 0x7f35d814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35dc848700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35d806dee0 0x7f35d80c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36550/0 (socket says 172.21.15.144:36550) 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35dd049700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35d8160540 0x7f35d80b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.686+0000 7f35dc848700 1 -- 172.21.15.144:0/4089737591 learned_addr learned my addr 172.21.15.144:0/4089737591 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d7fff700 1 -- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35d806dee0 msgr2=0x7f35d80c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d7fff700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35d806dee0 0x7f35d80c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d7fff700 1 -- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35d8160540 msgr2=0x7f35d80b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d7fff700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35d8160540 0x7f35d80b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d7fff700 1 -- 172.21.15.144:0/4089737591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0094a0 con 0x7f35d8071040 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d7fff700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8071040 0x7f35d814c3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c440 tx=0x7f35c400c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40052d0 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c4005430 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d0009b30 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4006aa0 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.687+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d815ccd0 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.688+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d815d900 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.688+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f35c4010050 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.689+0000 7f35d57fa700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f35bc0682a0 0x7f35bc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.689+0000 7f35dc848700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f35bc0682a0 0x7f35bc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.689+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f35c4084950 con 0x7f35d8071040 2023-11-28T11:51:13.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.689+0000 7f35dc848700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f35bc0682a0 0x7f35bc06a750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009dd0 tx=0x7f35cc009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:13.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.693+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35c404a5e0 con 0x7f35d8071040 2023-11-28T11:51:13.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.875+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35d806a270 con 0x7f35d8071040 2023-11-28T11:51:13.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.876+0000 7f35d57fa700 1 -- 172.21.15.144:0/4089737591 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35c40513e0 con 0x7f35d8071040 2023-11-28T11:51:13.877 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f35bc0682a0 msgr2=0x7f35bc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f35bc0682a0 0x7f35bc06a750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009dd0 tx=0x7f35cc009210 comp rx=0 tx=0).stop 2023-11-28T11:51:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8071040 msgr2=0x7f35d814c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8071040 0x7f35d814c3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c440 tx=0x7f35c400c750 comp rx=0 tx=0).stop 2023-11-28T11:51:13.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 shutdown_connections 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35d8160540 0x7f35d80b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f35bc0682a0 0x7f35bc06a750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35d806dee0 0x7f35d80c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 --2- 172.21.15.144:0/4089737591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35d8071040 0x7f35d814c3d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 >> 172.21.15.144:0/4089737591 conn(0x7f35d80d0000 msgr2=0x7f35d815e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 shutdown_connections 2023-11-28T11:51:13.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:13.878+0000 7f35deaac700 1 -- 172.21.15.144:0/4089737591 wait complete. 2023-11-28T11:51:13.891 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:51:14.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.009+0000 7fa7ced9a700 1 Processor -- start 2023-11-28T11:51:14.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7ced9a700 1 -- start start 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7ced9a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cd8c0 0x7fa7c8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7ced9a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa7c806f850 0x7fa7c806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7ced9a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c813ff00 con 0x7fa7c80cdc90 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7ced9a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c81404c0 con 0x7fa7c806f850 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7ced9a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c815aa80 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.010+0000 7fa7c7fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa7c80cdc90 0x7fa7c80b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:50996/0 (socket says 172.21.15.144:50996) 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7c7fff700 1 -- 172.21.15.144:0/3162477092 learned_addr learned my addr 172.21.15.144:0/3162477092 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7ccb36700 1 --2- 172.21.15.144:0/3162477092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cd8c0 0x7fa7c8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7cd337700 1 --2- 172.21.15.144:0/3162477092 >> v2:172.21.15.177:3300/0 conn(0x7fa7c806f850 0x7fa7c806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7c77fe700 1 -- 172.21.15.144:0/3162477092 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226735444 0 0) 0x7fa7c813ff00 con 0x7fa7c80cdc90 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7c77fe700 1 -- 172.21.15.144:0/3162477092 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b80035b0 con 0x7fa7c80cdc90 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7ccb36700 1 -- 172.21.15.144:0/3162477092 >> v2:172.21.15.177:3300/0 conn(0x7fa7c806f850 msgr2=0x7fa7c806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7ccb36700 1 --2- 172.21.15.144:0/3162477092 >> v2:172.21.15.177:3300/0 conn(0x7fa7c806f850 0x7fa7c806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7ccb36700 1 -- 172.21.15.144:0/3162477092 >> v1:172.21.15.144:6789/0 conn(0x7fa7c80cdc90 legacy=0x7fa7c80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7ccb36700 1 -- 172.21.15.144:0/3162477092 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c80b3040 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7cd337700 1 --2- 172.21.15.144:0/3162477092 >> v2:172.21.15.177:3300/0 conn(0x7fa7c806f850 0x7fa7c806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7ccb36700 1 --2- 172.21.15.144:0/3162477092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cd8c0 0x7fa7c8140950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009820 tx=0x7fa7bc009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7a2eb78b3a0914 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7c77fe700 1 -- 172.21.15.144:0/3162477092 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc005e30 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7c77fe700 1 -- 172.21.15.144:0/3162477092 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc006030 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.011+0000 7fa7c77fe700 1 -- 172.21.15.144:0/3162477092 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0063c0 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 -- 172.21.15.144:0/3162477092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cd8c0 msgr2=0x7fa7c8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/3162477092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cd8c0 0x7fa7c8140950 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009820 tx=0x7fa7bc009b30 comp rx=0 tx=0).stop 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 -- 172.21.15.144:0/3162477092 shutdown_connections 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/3162477092 >> v2:172.21.15.177:3300/0 conn(0x7fa7c806f850 0x7fa7c806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/3162477092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cd8c0 0x7fa7c8140950 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 -- 172.21.15.144:0/3162477092 >> 172.21.15.144:0/3162477092 conn(0x7fa7c806aed0 msgr2=0x7fa7c806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 -- 172.21.15.144:0/3162477092 shutdown_connections 2023-11-28T11:51:14.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.012+0000 7fa7ced9a700 1 -- 172.21.15.144:0/3162477092 wait complete. 2023-11-28T11:51:14.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.013+0000 7fa7ced9a700 1 Processor -- start 2023-11-28T11:51:14.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.013+0000 7fa7ced9a700 1 -- start start 2023-11-28T11:51:14.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ced9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c806f850 0x7fa7c8157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ced9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 0x7fa7c80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ced9a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cdc90 0x7fa7c80b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ced9a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80b5f00 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ced9a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c8142fe0 con 0x7fa7c806f850 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ced9a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c8157c60 con 0x7fa7c80cdc90 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 0x7fa7c80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 0x7fa7c80b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34292/0 (socket says 172.21.15.144:34292) 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 -- 172.21.15.144:0/1553518016 learned_addr learned my addr 172.21.15.144:0/1553518016 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7ccb36700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c806f850 0x7fa7c8157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7cd337700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cdc90 0x7fa7c80b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 -- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cdc90 msgr2=0x7fa7c80b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cdc90 0x7fa7c80b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 -- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c806f850 msgr2=0x7fa7c8157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c806f850 0x7fa7c8157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 -- 172.21.15.144:0/1553518016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0009a70 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.014+0000 7fa7c7fff700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 0x7fa7c80b4710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005910 tx=0x7fa7b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.015+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4005d70 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.015+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b40070d0 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.015+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc009430 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.015+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4007440 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.015+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c80b35f0 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.016+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c80b3310 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.016+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fa7b4019030 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.016+0000 7fa7c57fa700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7ac0682a0 0x7fa7ac06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.017+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7b4084ab0 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.017+0000 7fa7ccb36700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7ac0682a0 0x7fa7ac06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.017+0000 7fa7ccb36700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7ac0682a0 0x7fa7ac06a750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc000c00 tx=0x7fa7bc008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:14.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.021+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7b404b5e0 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.204+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa7c8140150 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.204+0000 7fa7c57fa700 1 -- 172.21.15.144:0/1553518016 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa7b4017090 con 0x7fa7c80cd8c0 2023-11-28T11:51:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:14.205 INFO:teuthology.orchestra.run.smithi144.stdout:[ 2023-11-28T11:51:14.205 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 1, 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": "rbd" 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 2, 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": ".mgr" 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:51:14.206 INFO:teuthology.orchestra.run.smithi144.stdout:] 2023-11-28T11:51:14.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.206+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7ac0682a0 msgr2=0x7fa7ac06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.206+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7ac0682a0 0x7fa7ac06a750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc000c00 tx=0x7fa7bc008070 comp rx=0 tx=0).stop 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.206+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 msgr2=0x7fa7c80b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.206+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 0x7fa7c80b4710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005910 tx=0x7fa7b400c860 comp rx=0 tx=0).stop 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 shutdown_connections 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7ac0682a0 0x7fa7ac06a750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80cdc90 0x7fa7c80b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c806f850 0x7fa7c8157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 --2- 172.21.15.144:0/1553518016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c80cd8c0 0x7fa7c80b4710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 >> 172.21.15.144:0/1553518016 conn(0x7fa7c806aed0 msgr2=0x7fa7c8159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 shutdown_connections 2023-11-28T11:51:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.207+0000 7fa7ced9a700 1 -- 172.21.15.144:0/1553518016 wait complete. 2023-11-28T11:51:14.219 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:51:14.219 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:51:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.342+0000 7eff5189f700 1 Processor -- start 2023-11-28T11:51:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.342+0000 7eff5189f700 1 -- start start 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.342+0000 7eff5189f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff4c161200 0x7eff4c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.342+0000 7eff5189f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4c160d70 con 0x7eff4c15a760 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.342+0000 7eff5189f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4c0b5390 con 0x7eff4c0cded0 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.342+0000 7eff5189f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c15ab30 con 0x7eff4c161200 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff4c161200 0x7eff4c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff4c161200 0x7eff4c15dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34302/0 (socket says 172.21.15.144:34302) 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 -- 172.21.15.144:0/4178073542 learned_addr learned my addr 172.21.15.144:0/4178073542 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 -- 172.21.15.144:0/4178073542 >> v1:172.21.15.177:6789/0 conn(0x7eff4c0cded0 legacy=0x7eff4c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 -- 172.21.15.144:0/4178073542 >> v1:172.21.15.144:6789/0 conn(0x7eff4c15a760 legacy=0x7eff4c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 -- 172.21.15.144:0/4178073542 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0b5670 con 0x7eff4c161200 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff4b7fe700 1 --2- 172.21.15.144:0/4178073542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c161200 0x7eff4c15dc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff44009e50 tx=0x7eff4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90faa6e02c15f68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff49ffb700 1 -- 172.21.15.144:0/4178073542 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400e070 con 0x7eff4c161200 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff49ffb700 1 -- 172.21.15.144:0/4178073542 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4400b9c0 con 0x7eff4c161200 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.343+0000 7eff49ffb700 1 -- 172.21.15.144:0/4178073542 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400bd70 con 0x7eff4c161200 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 -- 172.21.15.144:0/4178073542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c161200 msgr2=0x7eff4c15dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 --2- 172.21.15.144:0/4178073542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c161200 0x7eff4c15dc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff44009e50 tx=0x7eff4400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 -- 172.21.15.144:0/4178073542 shutdown_connections 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 --2- 172.21.15.144:0/4178073542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c161200 0x7eff4c15dc30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 -- 172.21.15.144:0/4178073542 >> 172.21.15.144:0/4178073542 conn(0x7eff4c072620 msgr2=0x7eff4c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 -- 172.21.15.144:0/4178073542 shutdown_connections 2023-11-28T11:51:14.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.344+0000 7eff5189f700 1 -- 172.21.15.144:0/4178073542 wait complete. 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.345+0000 7eff5189f700 1 Processor -- start 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.345+0000 7eff5189f700 1 -- start start 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff5189f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff4c0cded0 0x7eff4c1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff5189f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 0x7eff4c0b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff5189f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff4c161200 0x7eff4c0b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff5189f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0c8ff0 con 0x7eff4c15a760 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff5189f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0c91d0 con 0x7eff4c0cded0 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff5189f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0c94b0 con 0x7eff4c161200 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff4a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 0x7eff4c0b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff4a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 0x7eff4c0b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34306/0 (socket says 172.21.15.144:34306) 2023-11-28T11:51:14.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.346+0000 7eff4a7fc700 1 -- 172.21.15.144:0/3319569800 learned_addr learned my addr 172.21.15.144:0/3319569800 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4b7fe700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff4c161200 0x7eff4c0b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4affd700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff4c0cded0 0x7eff4c1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4a7fc700 1 -- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff4c161200 msgr2=0x7eff4c0b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4a7fc700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff4c161200 0x7eff4c0b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4a7fc700 1 -- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff4c0cded0 msgr2=0x7eff4c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4a7fc700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff4c0cded0 0x7eff4c1557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4a7fc700 1 -- 172.21.15.144:0/3319569800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44009b30 con 0x7eff4c15a760 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff4a7fc700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 0x7eff4c0b7f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c4b0 tx=0x7eff3c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0052d0 con 0x7eff4c15a760 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c005430 con 0x7eff4c15a760 2023-11-28T11:51:14.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c160a00 con 0x7eff4c15a760 2023-11-28T11:51:14.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0038c0 con 0x7eff4c15a760 2023-11-28T11:51:14.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.347+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c160720 con 0x7eff4c15a760 2023-11-28T11:51:14.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.348+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4c0b4bf0 con 0x7eff4c15a760 2023-11-28T11:51:14.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.348+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7eff3c019030 con 0x7eff4c15a760 2023-11-28T11:51:14.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.349+0000 7eff33fff700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7eff380682a0 0x7eff3806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:14.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.349+0000 7eff4affd700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7eff380682a0 0x7eff3806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:14.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.349+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7eff3c084b00 con 0x7eff4c15a760 2023-11-28T11:51:14.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.349+0000 7eff4affd700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7eff380682a0 0x7eff3806a750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff40007fd0 tx=0x7eff40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:14.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.353+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff3c051700 con 0x7eff4c15a760 2023-11-28T11:51:14.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:14.539+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff4c0c9970 con 0x7eff4c15a760 2023-11-28T11:51:15.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.466+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff3c010050 con 0x7eff4c15a760 2023-11-28T11:51:15.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.468+0000 7eff33fff700 1 -- 172.21.15.144:0/3319569800 <== mon.0 v2:172.21.15.144:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff3c051700 con 0x7eff4c15a760 2023-11-28T11:51:15.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.472+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7eff380682a0 msgr2=0x7eff3806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:15.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.472+0000 7eff5189f700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7eff380682a0 0x7eff3806a750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff40007fd0 tx=0x7eff40009040 comp rx=0 tx=0).stop 2023-11-28T11:51:15.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.472+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 msgr2=0x7eff4c0b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:15.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.472+0000 7eff5189f700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 0x7eff4c0b7f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00c4b0 tx=0x7eff3c00c7c0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 shutdown_connections 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7eff380682a0 0x7eff3806a750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff4c161200 0x7eff4c0b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff4c0cded0 0x7eff4c1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 --2- 172.21.15.144:0/3319569800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff4c15a760 0x7eff4c0b7f70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 >> 172.21.15.144:0/3319569800 conn(0x7eff4c072620 msgr2=0x7eff4c15db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 shutdown_connections 2023-11-28T11:51:15.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.473+0000 7eff5189f700 1 -- 172.21.15.144:0/3319569800 wait complete. 2023-11-28T11:51:15.475 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' created 2023-11-28T11:51:15.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:51:15.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 Processor -- start 2023-11-28T11:51:15.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 -- start start 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3fdc0cdbd0 0x7f3fdc071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3fdc071580 0x7f3fdc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc071980 con 0x7f3fdc154e70 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc0ce050 con 0x7f3fdc071580 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.621+0000 7f3fe31a3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc155240 con 0x7f3fdc0cdbd0 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3fdc071580 0x7f3fdc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe0f3f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3fdc154e70 0x7f3fdc0cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51024/0 (socket says 172.21.15.144:51024) 2023-11-28T11:51:15.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3fdc071580 0x7f3fdc15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34316/0 (socket says 172.21.15.144:34316) 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe0f3f700 1 -- 172.21.15.144:0/3254421018 learned_addr learned my addr 172.21.15.144:0/3254421018 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fdbfff700 1 --2- 172.21.15.144:0/3254421018 >> v2:172.21.15.177:3300/0 conn(0x7f3fdc0cdbd0 0x7f3fdc071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 -- 172.21.15.144:0/3254421018 >> v2:172.21.15.177:3300/0 conn(0x7f3fdc0cdbd0 msgr2=0x7f3fdc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 --2- 172.21.15.144:0/3254421018 >> v2:172.21.15.177:3300/0 conn(0x7f3fdc0cdbd0 0x7f3fdc071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 -- 172.21.15.144:0/3254421018 >> v1:172.21.15.144:6789/0 conn(0x7f3fdc154e70 legacy=0x7f3fdc0cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 -- 172.21.15.144:0/3254421018 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc15b100 con 0x7f3fdc071580 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fdbfff700 1 --2- 172.21.15.144:0/3254421018 >> v2:172.21.15.177:3300/0 conn(0x7f3fdc0cdbd0 0x7f3fdc071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fe1740700 1 --2- 172.21.15.144:0/3254421018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc071580 0x7f3fdc15e680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0c7060 tx=0x7f3fd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bec87117ba75bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fdb7fe700 1 -- 172.21.15.144:0/3254421018 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400e070 con 0x7f3fdc071580 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fdb7fe700 1 -- 172.21.15.144:0/3254421018 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd400bbd0 con 0x7f3fdc071580 2023-11-28T11:51:15.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.622+0000 7f3fdb7fe700 1 -- 172.21.15.144:0/3254421018 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4012430 con 0x7f3fdc071580 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 -- 172.21.15.144:0/3254421018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc071580 msgr2=0x7f3fdc15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/3254421018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc071580 0x7f3fdc15e680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0c7060 tx=0x7f3fd400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 -- 172.21.15.144:0/3254421018 shutdown_connections 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/3254421018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc071580 0x7f3fdc15e680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/3254421018 >> v2:172.21.15.177:3300/0 conn(0x7f3fdc0cdbd0 0x7f3fdc071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 -- 172.21.15.144:0/3254421018 >> 172.21.15.144:0/3254421018 conn(0x7f3fdc0d0000 msgr2=0x7f3fdc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 -- 172.21.15.144:0/3254421018 shutdown_connections 2023-11-28T11:51:15.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.623+0000 7f3fe31a3700 1 -- 172.21.15.144:0/3254421018 wait complete. 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.624+0000 7f3fe31a3700 1 Processor -- start 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.624+0000 7f3fe31a3700 1 -- start start 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe31a3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fdc071580 0x7f3fdc0c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe31a3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe31a3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 0x7f3fdc0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe31a3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0b5a80 con 0x7f3fdc154e70 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe31a3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c6530 con 0x7f3fdc0cdbd0 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe31a3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0b3fa0 con 0x7f3fdc071580 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 0x7f3fdc0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 0x7f3fdc0cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34322/0 (socket says 172.21.15.144:34322) 2023-11-28T11:51:15.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 -- 172.21.15.144:0/2761027286 learned_addr learned my addr 172.21.15.144:0/2761027286 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 -- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fdc071580 msgr2=0x7f3fdc0c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fdbfff700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe0f3f700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fdc071580 0x7f3fdc0c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fdc071580 0x7f3fdc0c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 -- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fdc0cdbd0 msgr2=0x7f3fdc159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 -- 172.21.15.144:0/2761027286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4009a70 con 0x7f3fdc154e70 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.625+0000 7f3fe1740700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 0x7f3fdc0cbf30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4014040 tx=0x7f3fd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.626+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4012430 con 0x7f3fdc154e70 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.626+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4020070 con 0x7f3fdc154e70 2023-11-28T11:51:15.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.626+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc15b6c0 con 0x7f3fdc154e70 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.626+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4004280 con 0x7f3fdc154e70 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.626+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc15b3e0 con 0x7f3fdc154e70 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.626+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc15b9a0 con 0x7f3fdc154e70 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.627+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fd402e070 con 0x7f3fdc154e70 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.627+0000 7f3fd97fa700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3fc00682f0 0x7f3fc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.627+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3fd4022070 con 0x7f3fdc154e70 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.627+0000 7f3fe0f3f700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3fc00682f0 0x7f3fc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:15.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.628+0000 7f3fe0f3f700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3fc00682f0 0x7f3fc006a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005910 tx=0x7f3fd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:15.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.632+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fd40542d0 con 0x7f3fdc154e70 2023-11-28T11:51:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:15.815+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3fdc0b5bc0 con 0x7f3fdc154e70 2023-11-28T11:51:16.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.468+0000 7f3fd97fa700 1 -- 172.21.15.144:0/2761027286 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3fdc0b5bc0 con 0x7f3fdc154e70 2023-11-28T11:51:16.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3fc00682f0 msgr2=0x7f3fc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3fc00682f0 0x7f3fc006a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0005910 tx=0x7f3fd0009040 comp rx=0 tx=0).stop 2023-11-28T11:51:16.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 msgr2=0x7f3fdc0cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 0x7f3fdc0cbf30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4014040 tx=0x7f3fd4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 shutdown_connections 2023-11-28T11:51:16.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fdc154e70 0x7f3fdc0cbf30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3fc00682f0 0x7f3fc006a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3fdc071580 0x7f3fdc0c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 --2- 172.21.15.144:0/2761027286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fdc0cdbd0 0x7f3fdc159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.475+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 >> 172.21.15.144:0/2761027286 conn(0x7f3fdc0d0000 msgr2=0x7f3fdc0ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:16.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.478+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 shutdown_connections 2023-11-28T11:51:16.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.478+0000 7f3fe31a3700 1 -- 172.21.15.144:0/2761027286 wait complete. 2023-11-28T11:51:16.481 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' created 2023-11-28T11:51:16.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:51:16.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.627+0000 7fc9bdcd5700 1 Processor -- start 2023-11-28T11:51:16.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.627+0000 7fc9bdcd5700 1 -- start start 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9bdcd5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9b806d4f0 0x7fc9b80cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9bdcd5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b80cda60 con 0x7fc9b8159650 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9bdcd5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b815e050 con 0x7fc9b815dae0 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9bdcd5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b80ce630 con 0x7fc9b806d4f0 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b6ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9b806d4f0 0x7fc9b80cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b77fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc9b8159650 0x7fc9b806cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51038/0 (socket says 172.21.15.144:51038) 2023-11-28T11:51:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b6ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9b806d4f0 0x7fc9b80cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34328/0 (socket says 172.21.15.144:34328) 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b77fe700 1 -- 172.21.15.144:0/343537516 learned_addr learned my addr 172.21.15.144:0/343537516 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/343537516 >> v1:172.21.15.177:6789/0 conn(0x7fc9b815dae0 legacy=0x7fc9b80c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/343537516 >> v1:172.21.15.144:6789/0 conn(0x7fc9b8159650 legacy=0x7fc9b806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/343537516 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b80cdc90 con 0x7fc9b806d4f0 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b67fc700 1 -- 172.21.15.144:0/343537516 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391663075 0 0) 0x7fc9b80cda60 con 0x7fc9b8159650 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b6ffd700 1 --2- 172.21.15.144:0/343537516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b806d4f0 0x7fc9b80cea50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800bae0 tx=0x7fc9a800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e18f648993efd0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b67fc700 1 -- 172.21.15.144:0/343537516 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800dcf0 con 0x7fc9b806d4f0 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b67fc700 1 -- 172.21.15.144:0/343537516 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a800de50 con 0x7fc9b806d4f0 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.628+0000 7fc9b67fc700 1 -- 172.21.15.144:0/343537516 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8004250 con 0x7fc9b806d4f0 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/343537516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b806d4f0 msgr2=0x7fc9b80cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/343537516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b806d4f0 0x7fc9b80cea50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800bae0 tx=0x7fc9a800bdf0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/343537516 shutdown_connections 2023-11-28T11:51:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/343537516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b806d4f0 0x7fc9b80cea50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/343537516 >> 172.21.15.144:0/343537516 conn(0x7fc9b80b19f0 msgr2=0x7fc9b80b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/343537516 shutdown_connections 2023-11-28T11:51:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.629+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/343537516 wait complete. 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.631+0000 7fc9bdcd5700 1 Processor -- start 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.631+0000 7fc9bdcd5700 1 -- start start 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9b806d4f0 0x7fc9b80b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 0x7fc9b80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b815dae0 0x7fc9b81f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b80cd0b0 con 0x7fc9b8159650 2023-11-28T11:51:16.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b80cc8c0 con 0x7fc9b815dae0 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b81f67d0 con 0x7fc9b806d4f0 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 0x7fc9b80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 0x7fc9b80cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34334/0 (socket says 172.21.15.144:34334) 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/4047548995 learned_addr learned my addr 172.21.15.144:0/4047548995 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b7fff700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b815dae0 0x7fc9b81f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9b806d4f0 msgr2=0x7fc9b80b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9b806d4f0 0x7fc9b80b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b815dae0 msgr2=0x7fc9b81f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b815dae0 0x7fc9b81f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 -- 172.21.15.144:0/4047548995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a800b740 con 0x7fc9b8159650 2023-11-28T11:51:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9b6ffd700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 0x7fc9b80cc380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b2c0 tx=0x7fc9a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8004230 con 0x7fc9b8159650 2023-11-28T11:51:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8010470 con 0x7fc9b8159650 2023-11-28T11:51:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b80ce250 con 0x7fc9b8159650 2023-11-28T11:51:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8010800 con 0x7fc9b8159650 2023-11-28T11:51:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.632+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b80cdf70 con 0x7fc9b8159650 2023-11-28T11:51:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.634+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9a802b3f0 con 0x7fc9b8159650 2023-11-28T11:51:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.634+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b80c4e30 con 0x7fc9b8159650 2023-11-28T11:51:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.634+0000 7fc99ffff700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9a40682f0 0x7fc9a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.634+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc9a80873c0 con 0x7fc9b8159650 2023-11-28T11:51:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.634+0000 7fc9b77fe700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9a40682f0 0x7fc9a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.635+0000 7fc9b77fe700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9a40682f0 0x7fc9a406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0011e0 tx=0x7fc9ac0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.639+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9a8053ba0 con 0x7fc9b8159650 2023-11-28T11:51:16.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.865+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc9b8068690 con 0x7fc9b8159650 2023-11-28T11:51:16.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.871+0000 7fc99ffff700 1 -- 172.21.15.144:0/4047548995 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc9a80571c0 con 0x7fc9b8159650 2023-11-28T11:51:16.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.874+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9a40682f0 msgr2=0x7fc9a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.874+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9a40682f0 0x7fc9a406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0011e0 tx=0x7fc9ac0079b0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.874+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 msgr2=0x7fc9b80cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:16.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.874+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 0x7fc9b80cc380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b2c0 tx=0x7fc9a8004750 comp rx=0 tx=0).stop 2023-11-28T11:51:16.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.875+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 shutdown_connections 2023-11-28T11:51:16.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.875+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc9a40682f0 0x7fc9a406a7a0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac0011e0 tx=0x7fc9ac0079b0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.875+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b815dae0 0x7fc9b81f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.875+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9b806d4f0 0x7fc9b80b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:16.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.875+0000 7fc9bdcd5700 1 --2- 172.21.15.144:0/4047548995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9b8159650 0x7fc9b80cc380 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800b2c0 tx=0x7fc9a8004750 comp rx=0 tx=0).stop 2023-11-28T11:51:16.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.875+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 >> 172.21.15.144:0/4047548995 conn(0x7fc9b80b19f0 msgr2=0x7fc9b80c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:16.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.882+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 shutdown_connections 2023-11-28T11:51:16.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:16.882+0000 7fc9bdcd5700 1 -- 172.21.15.144:0/4047548995 wait complete. 2023-11-28T11:51:16.885 INFO:teuthology.orchestra.run.smithi144.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T11:51:16.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:17.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.021+0000 7fb7d8ba0700 1 Processor -- start 2023-11-28T11:51:17.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.021+0000 7fb7d8ba0700 1 -- start start 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d8ba0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb7d406ec90 0x7fb7d4065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d8ba0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb7d40cb0e0 0x7fb7d406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d8ba0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d4066580 con 0x7fb7d40cec70 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d8ba0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d40cded0 con 0x7fb7d406ec90 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d8ba0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d40ce1b0 con 0x7fb7d40cb0e0 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb7d406ec90 0x7fb7d4065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb7d406ec90 0x7fb7d4065ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34340/0 (socket says 172.21.15.144:34340) 2023-11-28T11:51:17.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 -- 172.21.15.144:0/2366529280 learned_addr learned my addr 172.21.15.144:0/2366529280 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d2d9d700 1 --2- 172.21.15.144:0/2366529280 >> v2:172.21.15.177:3300/0 conn(0x7fb7d40cb0e0 0x7fb7d406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 -- 172.21.15.144:0/2366529280 >> v2:172.21.15.177:3300/0 conn(0x7fb7d40cb0e0 msgr2=0x7fb7d406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 --2- 172.21.15.144:0/2366529280 >> v2:172.21.15.177:3300/0 conn(0x7fb7d40cb0e0 0x7fb7d406d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 -- 172.21.15.144:0/2366529280 >> v1:172.21.15.177:6789/0 conn(0x7fb7d40cec70 legacy=0x7fb7d40ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.022+0000 7fb7d259c700 1 -- 172.21.15.144:0/2366529280 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d40ce490 con 0x7fb7d406ec90 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d259c700 1 --2- 172.21.15.144:0/2366529280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d406ec90 0x7fb7d4065ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb7d406c530 tx=0x7fb7c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27e10507acc9ca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d159a700 1 -- 172.21.15.144:0/2366529280 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8004230 con 0x7fb7d406ec90 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d159a700 1 -- 172.21.15.144:0/2366529280 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8004390 con 0x7fb7d406ec90 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d159a700 1 -- 172.21.15.144:0/2366529280 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80036a0 con 0x7fb7d406ec90 2023-11-28T11:51:17.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2366529280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d406ec90 msgr2=0x7fb7d4065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2366529280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d406ec90 0x7fb7d4065ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb7d406c530 tx=0x7fb7c8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2366529280 shutdown_connections 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2366529280 >> v2:172.21.15.177:3300/0 conn(0x7fb7d40cb0e0 0x7fb7d406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2366529280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d406ec90 0x7fb7d4065ef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2366529280 >> 172.21.15.144:0/2366529280 conn(0x7fb7d40b12b0 msgr2=0x7fb7d40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2366529280 shutdown_connections 2023-11-28T11:51:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.023+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2366529280 wait complete. 2023-11-28T11:51:17.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.025+0000 7fb7d8ba0700 1 Processor -- start 2023-11-28T11:51:17.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.025+0000 7fb7d8ba0700 1 -- start start 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d8ba0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d406ec90 0x7fb7d4148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d8ba0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7d40cb0e0 0x7fb7d4149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d8ba0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 0x7fb7d40c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d8ba0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d41e2ae0 con 0x7fb7d40cec70 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d8ba0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d41e21d0 con 0x7fb7d406ec90 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d8ba0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4158670 con 0x7fb7d40cb0e0 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 0x7fb7d40c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 0x7fb7d40c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34350/0 (socket says 172.21.15.144:34350) 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 -- 172.21.15.144:0/2317821099 learned_addr learned my addr 172.21.15.144:0/2317821099 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d259c700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d406ec90 0x7fb7d4148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d1d9b700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7d40cb0e0 0x7fb7d4149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 -- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7d40cb0e0 msgr2=0x7fb7d4149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7d40cb0e0 0x7fb7d4149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 -- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d406ec90 msgr2=0x7fb7d4148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d406ec90 0x7fb7d4148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 -- 172.21.15.144:0/2317821099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c8009550 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d1d9b700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7d40cb0e0 0x7fb7d4149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.026+0000 7fb7d2d9d700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 0x7fb7d40c8aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc007e90 tx=0x7fb7cc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.027+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00ede0 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.027+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc014420 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.027+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc0147d0 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.027+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d40cb6c0 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.027+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d40ce770 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.028+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d406f060 con 0x7fb7d40cec70 2023-11-28T11:51:17.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.028+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb7cc012050 con 0x7fb7d40cec70 2023-11-28T11:51:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.028+0000 7fb7c2ffd700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7b40682f0 0x7fb7b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.028+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb7cc086210 con 0x7fb7d40cec70 2023-11-28T11:51:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.028+0000 7fb7d259c700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7b40682f0 0x7fb7b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.029+0000 7fb7d259c700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7b40682f0 0x7fb7b406a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009130 tx=0x7fb7c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.033+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7cc052a70 con 0x7fb7d40cec70 2023-11-28T11:51:17.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.215+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7d41e2c20 con 0x7fb7d40cec70 2023-11-28T11:51:17.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.216+0000 7fb7c2ffd700 1 -- 172.21.15.144:0/2317821099 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17910 (secure 0 0 0) 0x7fb7d41e2c20 con 0x7fb7d40cec70 2023-11-28T11:51:17.217 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:17.218 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":21,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:16.866531+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:17.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7b40682f0 msgr2=0x7fb7b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7b40682f0 0x7fb7b406a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009130 tx=0x7fb7c8008030 comp rx=0 tx=0).stop 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 msgr2=0x7fb7d40c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 0x7fb7d40c8aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc007e90 tx=0x7fb7cc00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 shutdown_connections 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d40cec70 0x7fb7d40c8aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7b40682f0 0x7fb7b406a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d406ec90 0x7fb7d4148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 --2- 172.21.15.144:0/2317821099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7d40cb0e0 0x7fb7d4149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.218+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 >> 172.21.15.144:0/2317821099 conn(0x7fb7d40b12b0 msgr2=0x7fb7d40b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.219+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 shutdown_connections 2023-11-28T11:51:17.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.219+0000 7fb7d8ba0700 1 -- 172.21.15.144:0/2317821099 wait complete. 2023-11-28T11:51:17.232 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.351+0000 7f8b075dd700 1 Processor -- start 2023-11-28T11:51:17.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.351+0000 7f8b075dd700 1 -- start start 2023-11-28T11:51:17.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b075dd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8b0015cef0 0x7f8b000af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b075dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b0006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b075dd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8b0006b8c0 0x7f8b00071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b075dd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0006bd40 con 0x7f8b0015cef0 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b075dd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000cc5f0 con 0x7f8b0006b8c0 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b075dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0015d470 con 0x7f8b000cbdf0 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8b0015cef0 0x7f8b000af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8b0015cef0 0x7f8b000af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34354/0 (socket says 172.21.15.144:34354) 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 -- 172.21.15.144:0/3462394883 learned_addr learned my addr 172.21.15.144:0/3462394883 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b04b78700 1 --2- 172.21.15.144:0/3462394883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b0006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05b7a700 1 --2- 172.21.15.144:0/3462394883 >> v2:172.21.15.177:3300/0 conn(0x7f8b0006b8c0 0x7f8b00071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 -- 172.21.15.144:0/3462394883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 msgr2=0x7f8b0006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 --2- 172.21.15.144:0/3462394883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b0006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 -- 172.21.15.144:0/3462394883 >> v2:172.21.15.177:3300/0 conn(0x7f8b0006b8c0 msgr2=0x7f8b00071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 --2- 172.21.15.144:0/3462394883 >> v2:172.21.15.177:3300/0 conn(0x7f8b0006b8c0 0x7f8b00071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 -- 172.21.15.144:0/3462394883 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0015d750 con 0x7f8b0015cef0 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05b7a700 1 --2- 172.21.15.144:0/3462394883 >> v2:172.21.15.177:3300/0 conn(0x7f8b0006b8c0 0x7f8b00071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8b05379700 1 --2- 172.21.15.144:0/3462394883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0015cef0 0x7f8b000af400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b00073a70 tx=0x7f8af0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abf5af726df5c52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8af7fff700 1 -- 172.21.15.144:0/3462394883 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011830 con 0x7f8b0015cef0 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8af7fff700 1 -- 172.21.15.144:0/3462394883 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af0011990 con 0x7f8b0015cef0 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.352+0000 7f8af7fff700 1 -- 172.21.15.144:0/3462394883 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011d20 con 0x7f8b0015cef0 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 -- 172.21.15.144:0/3462394883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0015cef0 msgr2=0x7f8b000af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 --2- 172.21.15.144:0/3462394883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0015cef0 0x7f8b000af400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b00073a70 tx=0x7f8af0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 -- 172.21.15.144:0/3462394883 shutdown_connections 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 --2- 172.21.15.144:0/3462394883 >> v2:172.21.15.177:3300/0 conn(0x7f8b0006b8c0 0x7f8b00071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 --2- 172.21.15.144:0/3462394883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0015cef0 0x7f8b000af400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 --2- 172.21.15.144:0/3462394883 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b0006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 -- 172.21.15.144:0/3462394883 >> 172.21.15.144:0/3462394883 conn(0x7f8b000d0000 msgr2=0x7f8b000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:17.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 -- 172.21.15.144:0/3462394883 shutdown_connections 2023-11-28T11:51:17.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.353+0000 7f8b075dd700 1 -- 172.21.15.144:0/3462394883 wait complete. 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.354+0000 7f8b075dd700 1 Processor -- start 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.354+0000 7f8b075dd700 1 -- start start 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 0x7f8b0015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b000b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0015cef0 0x7f8b001f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b3060 con 0x7f8b0006b8c0 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0015ab40 con 0x7f8b0015cef0 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b4fe0 con 0x7f8b000cbdf0 2023-11-28T11:51:17.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 0x7f8b0015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 0x7f8b0015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34358/0 (socket says 172.21.15.144:34358) 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 -- 172.21.15.144:0/4013289192 learned_addr learned my addr 172.21.15.144:0/4013289192 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 -- 172.21.15.144:0/4013289192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 msgr2=0x7f8b000b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b000b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 -- 172.21.15.144:0/4013289192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0015cef0 msgr2=0x7f8b001f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0015cef0 0x7f8b001f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 -- 172.21.15.144:0/4013289192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af0009550 con 0x7f8b0006b8c0 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b05379700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 0x7f8b0015a600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8af0000c00 tx=0x7f8af0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00036a0 con 0x7f8b0006b8c0 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af001f070 con 0x7f8b0006b8c0 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0015dd10 con 0x7f8b0006b8c0 2023-11-28T11:51:17.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.355+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0003a40 con 0x7f8b0006b8c0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.356+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0015da30 con 0x7f8b0006b8c0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.357+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0015dff0 con 0x7f8b0006b8c0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.357+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8af000f050 con 0x7f8b0006b8c0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.357+0000 7f8af5ffb700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8ae40682f0 0x7f8ae406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.357+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8af0021070 con 0x7f8b0006b8c0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.357+0000 7f8b04b78700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8ae40682f0 0x7f8ae406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.358+0000 7f8b04b78700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8ae40682f0 0x7f8ae406a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006b20 tx=0x7f8afc006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.362+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8af0053dd0 con 0x7f8b0006b8c0 2023-11-28T11:51:17.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.547+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b000b2e30 con 0x7f8b0006b8c0 2023-11-28T11:51:17.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.548+0000 7f8af5ffb700 1 -- 172.21.15.144:0/4013289192 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+17910 (secure 0 0 0) 0x7f8b000b2e30 con 0x7f8b0006b8c0 2023-11-28T11:51:17.549 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:17.549 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":21,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:16.866531+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:17.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8ae40682f0 msgr2=0x7f8ae406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8ae40682f0 0x7f8ae406a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006b20 tx=0x7f8afc006ab0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 msgr2=0x7f8b0015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 0x7f8b0015a600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8af0000c00 tx=0x7f8af0004750 comp rx=0 tx=0).stop 2023-11-28T11:51:17.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 shutdown_connections 2023-11-28T11:51:17.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b0006b8c0 0x7f8b0015a600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8ae40682f0 0x7f8ae406a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0015cef0 0x7f8b001f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 --2- 172.21.15.144:0/4013289192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b000cbdf0 0x7f8b000b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.550+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 >> 172.21.15.144:0/4013289192 conn(0x7f8b000d0000 msgr2=0x7f8b000af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:17.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.551+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 shutdown_connections 2023-11-28T11:51:17.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.551+0000 7f8b075dd700 1 -- 172.21.15.144:0/4013289192 wait complete. 2023-11-28T11:51:17.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:51:17.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.686+0000 7f39da1af700 1 Processor -- start 2023-11-28T11:51:17.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.687+0000 7f39da1af700 1 -- start start 2023-11-28T11:51:17.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.687+0000 7f39da1af700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 0x7f39d40af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.687+0000 7f39da1af700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d40cc5f0 con 0x7f39d406ba90 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.687+0000 7f39da1af700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d415d470 con 0x7f39d40cbdf0 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.687+0000 7f39da1af700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d415d750 con 0x7f39d415cef0 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d3fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f39d406ba90 0x7f39d40ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51082/0 (socket says 172.21.15.144:51082) 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d3fff700 1 -- 172.21.15.144:0/3261969052 learned_addr learned my addr 172.21.15.144:0/3261969052 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 0x7f39d40af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212381182 0 0) 0x7f39d40cc5f0 con 0x7f39d406ba90 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39bc0035b0 con 0x7f39d406ba90 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973674594 0 0) 0x7f39d415d470 con 0x7f39d40cbdf0 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d40cc5f0 con 0x7f39d40cbdf0 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1332611472 0 0) 0x7f39bc0035b0 con 0x7f39d406ba90 2023-11-28T11:51:17.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 msgr2=0x7f39d40af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 0x7f39d40af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 >> v1:172.21.15.177:6789/0 conn(0x7f39d40cbdf0 legacy=0x7f39d406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d415da30 con 0x7f39d406ba90 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f39cc0019f0 con 0x7f39d406ba90 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 >> v1:172.21.15.144:6789/0 conn(0x7f39d406ba90 legacy=0x7f39d40ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39bc003bc0 0x7f39bc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 0x7f39bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc004d90 0x7f39bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d415d750 con 0x7f39bc003bc0 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc00bac0 con 0x7f39bc004d90 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc00bc30 con 0x7f39d415cef0 2023-11-28T11:51:17.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39bc003bc0 0x7f39bc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d2ffd700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 0x7f39bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d3fff700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc004d90 0x7f39bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 -- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 msgr2=0x7f39bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d415cef0 0x7f39bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 -- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc004d90 msgr2=0x7f39bc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc004d90 0x7f39bc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.688+0000 7f39d37fe700 1 -- 172.21.15.144:0/3261969052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0035b0 con 0x7f39bc003bc0 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39d3fff700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc004d90 0x7f39bc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:17.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39d37fe700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39bc003bc0 0x7f39bc003fb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009b60 tx=0x7f39bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaf80302d55473e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800e660 con 0x7f39bc003bc0 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c800e7c0 con 0x7f39bc003bc0 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39d27fc700 1 -- 172.21.15.144:0/3261969052 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800eb30 con 0x7f39bc003bc0 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39da1af700 1 -- 172.21.15.144:0/3261969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39bc003bc0 msgr2=0x7f39bc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39da1af700 1 --2- 172.21.15.144:0/3261969052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39bc003bc0 0x7f39bc003fb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009b60 tx=0x7f39bc005880 comp rx=0 tx=0).stop 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39d3fff700 1 -- 172.21.15.144:0/3261969052 reap_dead start 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39da1af700 1 -- 172.21.15.144:0/3261969052 shutdown_connections 2023-11-28T11:51:17.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39da1af700 1 -- 172.21.15.144:0/3261969052 >> 172.21.15.144:0/3261969052 conn(0x7f39d40d0000 msgr2=0x7f39d40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:17.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39da1af700 1 -- 172.21.15.144:0/3261969052 shutdown_connections 2023-11-28T11:51:17.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.689+0000 7f39da1af700 1 -- 172.21.15.144:0/3261969052 wait complete. 2023-11-28T11:51:17.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.690+0000 7f39da1af700 1 Processor -- start 2023-11-28T11:51:17.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 -- start start 2023-11-28T11:51:17.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc003bc0 0x7f39d4154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 0x7f39d406a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d406a7b0 0x7f39d406ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c6780 con 0x7f39d4154620 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41e4110 con 0x7f39bc003bc0 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.691+0000 7f39da1af700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41e3500 con 0x7f39d406a7b0 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 0x7f39d406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 0x7f39d406a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34380/0 (socket says 172.21.15.144:34380) 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 -- 172.21.15.144:0/558069004 learned_addr learned my addr 172.21.15.144:0/558069004 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d37fe700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc003bc0 0x7f39d4154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d3fff700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d406a7b0 0x7f39d406ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 -- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d406a7b0 msgr2=0x7f39d406ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d406a7b0 0x7f39d406ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 -- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc003bc0 msgr2=0x7f39d4154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc003bc0 0x7f39d4154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 -- 172.21.15.144:0/558069004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d415dff0 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d3fff700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d406a7b0 0x7f39d406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39d2ffd700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 0x7f39d406a270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39c400f7a0 tx=0x7f39c400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40052e0 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4005460 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d415dd10 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40038c0 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.692+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40af400 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.693+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d40d0e60 con 0x7f39d4154620 2023-11-28T11:51:17.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.693+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39c4003a20 con 0x7f39d4154620 2023-11-28T11:51:17.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.694+0000 7f39bbfff700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39c00682f0 0x7f39c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:17.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.694+0000 7f39d37fe700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39c00682f0 0x7f39c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:17.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.694+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39c4085c50 con 0x7f39d4154620 2023-11-28T11:51:17.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.694+0000 7f39d37fe700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39c00682f0 0x7f39c006a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39c8000c00 tx=0x7f39c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:17.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.698+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39c40524b0 con 0x7f39d4154620 2023-11-28T11:51:17.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:17.927+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f39d406e400 con 0x7f39d4154620 2023-11-28T11:51:18.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.879+0000 7f39bbfff700 1 -- 172.21.15.144:0/558069004 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f39c401a310 con 0x7f39d4154620 2023-11-28T11:51:18.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39c00682f0 msgr2=0x7f39c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:18.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39c00682f0 0x7f39c006a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39c8000c00 tx=0x7f39c8008030 comp rx=0 tx=0).stop 2023-11-28T11:51:18.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 msgr2=0x7f39d406a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:18.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 0x7f39d406a270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39c400f7a0 tx=0x7f39c400fb60 comp rx=0 tx=0).stop 2023-11-28T11:51:18.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 shutdown_connections 2023-11-28T11:51:18.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39c00682f0 0x7f39c006a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:18.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39d406a7b0 0x7f39d406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:18.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc003bc0 0x7f39d4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:18.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 --2- 172.21.15.144:0/558069004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39d4154620 0x7f39d406a270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:18.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 >> 172.21.15.144:0/558069004 conn(0x7f39d40d0000 msgr2=0x7f39d406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:18.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 shutdown_connections 2023-11-28T11:51:18.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:18.882+0000 7f39da1af700 1 -- 172.21.15.144:0/558069004 wait complete. 2023-11-28T11:51:18.898 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:51:18.898 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:51:19.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.025+0000 7f9237b3c700 1 Processor -- start 2023-11-28T11:51:19.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.026+0000 7f9237b3c700 1 -- start start 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.026+0000 7f9237b3c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f92300cd8c0 0x7f9230140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.026+0000 7f9237b3c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923013ff00 con 0x7f923006f850 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.026+0000 7f9237b3c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92301404c0 con 0x7f92300cdc90 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.026+0000 7f9237b3c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923015aa80 con 0x7f92300cd8c0 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92358d8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f92300cd8c0 0x7f9230140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92360d9700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f923006f850 0x7f923006fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51098/0 (socket says 172.21.15.144:51098) 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92360d9700 1 -- 172.21.15.144:0/392327444 learned_addr learned my addr 172.21.15.144:0/392327444 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92358d8700 1 -- 172.21.15.144:0/392327444 >> v1:172.21.15.177:6789/0 conn(0x7f92300cdc90 legacy=0x7f92300b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:19.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92358d8700 1 -- 172.21.15.144:0/392327444 >> v1:172.21.15.144:6789/0 conn(0x7f923006f850 legacy=0x7f923006fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:19.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92358d8700 1 -- 172.21.15.144:0/392327444 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92300b3040 con 0x7f92300cd8c0 2023-11-28T11:51:19.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92348d6700 1 -- 172.21.15.144:0/392327444 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144644487 0 0) 0x7f923013ff00 con 0x7f923006f850 2023-11-28T11:51:19.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92358d8700 1 --2- 172.21.15.144:0/392327444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cd8c0 0x7f9230140950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9220009820 tx=0x7f9220009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00bac9c3cb7710b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:19.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92348d6700 1 -- 172.21.15.144:0/392327444 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220005e30 con 0x7f92300cd8c0 2023-11-28T11:51:19.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92348d6700 1 -- 172.21.15.144:0/392327444 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9220004030 con 0x7f92300cd8c0 2023-11-28T11:51:19.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.027+0000 7f92348d6700 1 -- 172.21.15.144:0/392327444 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200043c0 con 0x7f92300cd8c0 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 -- 172.21.15.144:0/392327444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cd8c0 msgr2=0x7f9230140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 --2- 172.21.15.144:0/392327444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cd8c0 0x7f9230140950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9220009820 tx=0x7f9220009b30 comp rx=0 tx=0).stop 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 -- 172.21.15.144:0/392327444 shutdown_connections 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 --2- 172.21.15.144:0/392327444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cd8c0 0x7f9230140950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 -- 172.21.15.144:0/392327444 >> 172.21.15.144:0/392327444 conn(0x7f923006aed0 msgr2=0x7f923006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 -- 172.21.15.144:0/392327444 shutdown_connections 2023-11-28T11:51:19.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.028+0000 7f9237b3c700 1 -- 172.21.15.144:0/392327444 wait complete. 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.029+0000 7f9237b3c700 1 Processor -- start 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.029+0000 7f9237b3c700 1 -- start start 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f923006f850 0x7f9230142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92300cd8c0 0x7f9230141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 0x7f923013cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92301432d0 con 0x7f92300cdc90 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92301435b0 con 0x7f923006f850 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9230143890 con 0x7f92300cd8c0 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 0x7f923013cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 0x7f923013cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34396/0 (socket says 172.21.15.144:34396) 2023-11-28T11:51:19.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 -- 172.21.15.144:0/420782088 learned_addr learned my addr 172.21.15.144:0/420782088 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92358d8700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f923006f850 0x7f9230142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92350d7700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92300cd8c0 0x7f9230141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 -- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92300cd8c0 msgr2=0x7f9230141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92300cd8c0 0x7f9230141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 -- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f923006f850 msgr2=0x7f9230142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f923006f850 0x7f9230142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 -- 172.21.15.144:0/420782088 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220009430 con 0x7f92300cdc90 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92350d7700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92300cd8c0 0x7f9230141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92358d8700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f923006f850 0x7f9230142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92360d9700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 0x7f923013cf90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f922800cca0 tx=0x7f922800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922800cea0 con 0x7f92300cdc90 2023-11-28T11:51:19.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922800ade0 con 0x7f92300cdc90 2023-11-28T11:51:19.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92300b35f0 con 0x7f92300cdc90 2023-11-28T11:51:19.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.030+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228007310 con 0x7f92300cdc90 2023-11-28T11:51:19.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.031+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92300b3310 con 0x7f92300cdc90 2023-11-28T11:51:19.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.031+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9230140150 con 0x7f92300cdc90 2023-11-28T11:51:19.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.032+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f922801a030 con 0x7f92300cdc90 2023-11-28T11:51:19.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.032+0000 7f92267fc700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92140682f0 0x7f921406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:19.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.032+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9228085a50 con 0x7f92300cdc90 2023-11-28T11:51:19.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.032+0000 7f92358d8700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92140682f0 0x7f921406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:19.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.033+0000 7f92358d8700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92140682f0 0x7f921406a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9220014040 tx=0x7f9220008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:19.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.037+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92280522b0 con 0x7f92300cdc90 2023-11-28T11:51:19.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.266+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9230068690 con 0x7f92300cdc90 2023-11-28T11:51:19.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.880+0000 7f92267fc700 1 -- 172.21.15.144:0/420782088 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92280180c0 con 0x7f92300cdc90 2023-11-28T11:51:19.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92140682f0 msgr2=0x7f921406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:19.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92140682f0 0x7f921406a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9220014040 tx=0x7f9220008070 comp rx=0 tx=0).stop 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 msgr2=0x7f923013cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 0x7f923013cf90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f922800cca0 tx=0x7f922800a340 comp rx=0 tx=0).stop 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 shutdown_connections 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92300cdc90 0x7f923013cf90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92140682f0 0x7f921406a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f923006f850 0x7f9230142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 --2- 172.21.15.144:0/420782088 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92300cd8c0 0x7f9230141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.883+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 >> 172.21.15.144:0/420782088 conn(0x7f923006aed0 msgr2=0x7f923013e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.884+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 shutdown_connections 2023-11-28T11:51:19.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:19.884+0000 7f9237b3c700 1 -- 172.21.15.144:0/420782088 wait complete. 2023-11-28T11:51:19.898 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:51:19.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:20.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 Processor -- start 2023-11-28T11:51:20.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 -- start start 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420071040 0x7fb420071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb42006dee0 0x7fb42006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb420071950 con 0x7fb42006cdc0 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb420161550 con 0x7fb42006dee0 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.021+0000 7fb426999700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42015d060 con 0x7fb420071040 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb424f36700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb42006cdc0 0x7fb42006d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51116/0 (socket says 172.21.15.144:51116) 2023-11-28T11:51:20.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb424f36700 1 -- 172.21.15.144:0/3547809238 learned_addr learned my addr 172.21.15.144:0/3547809238 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41ffff700 1 --2- 172.21.15.144:0/3547809238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420071040 0x7fb420071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41f7fe700 1 --2- 172.21.15.144:0/3547809238 >> v2:172.21.15.177:3300/0 conn(0x7fb42006dee0 0x7fb42006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41effd700 1 -- 172.21.15.144:0/3547809238 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009810394 0 0) 0x7fb420071950 con 0x7fb42006cdc0 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41effd700 1 -- 172.21.15.144:0/3547809238 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c0035b0 con 0x7fb42006cdc0 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41ffff700 1 -- 172.21.15.144:0/3547809238 >> v2:172.21.15.177:3300/0 conn(0x7fb42006dee0 msgr2=0x7fb42006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41ffff700 1 --2- 172.21.15.144:0/3547809238 >> v2:172.21.15.177:3300/0 conn(0x7fb42006dee0 0x7fb42006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41ffff700 1 -- 172.21.15.144:0/3547809238 >> v1:172.21.15.144:6789/0 conn(0x7fb42006cdc0 legacy=0x7fb42006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41effd700 1 -- 172.21.15.144:0/3547809238 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 829807476 0 0) 0x7fb40c0035b0 con 0x7fb42006cdc0 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41ffff700 1 -- 172.21.15.144:0/3547809238 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42015d340 con 0x7fb420071040 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41ffff700 1 --2- 172.21.15.144:0/3547809238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420071040 0x7fb420071410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb410009840 tx=0x7fb410009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=720e0013a217bd5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41effd700 1 -- 172.21.15.144:0/3547809238 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4180019f0 con 0x7fb420071040 2023-11-28T11:51:20.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41effd700 1 -- 172.21.15.144:0/3547809238 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410004030 con 0x7fb420071040 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.022+0000 7fb41effd700 1 -- 172.21.15.144:0/3547809238 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100043c0 con 0x7fb420071040 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 -- 172.21.15.144:0/3547809238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420071040 msgr2=0x7fb420071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 --2- 172.21.15.144:0/3547809238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420071040 0x7fb420071410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb410009840 tx=0x7fb410009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 -- 172.21.15.144:0/3547809238 shutdown_connections 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 --2- 172.21.15.144:0/3547809238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb420071040 0x7fb420071410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 --2- 172.21.15.144:0/3547809238 >> v2:172.21.15.177:3300/0 conn(0x7fb42006dee0 0x7fb42006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 -- 172.21.15.144:0/3547809238 >> 172.21.15.144:0/3547809238 conn(0x7fb4200d0000 msgr2=0x7fb4200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 -- 172.21.15.144:0/3547809238 shutdown_connections 2023-11-28T11:51:20.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.023+0000 7fb426999700 1 -- 172.21.15.144:0/3547809238 wait complete. 2023-11-28T11:51:20.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.025+0000 7fb426999700 1 Processor -- start 2023-11-28T11:51:20.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.025+0000 7fb426999700 1 -- start start 2023-11-28T11:51:20.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 0x7fb4201efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42006dee0 0x7fb4201f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420071040 0x7fb4200ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4201f36a0 con 0x7fb42006cdc0 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb420159230 con 0x7fb420071040 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb420159510 con 0x7fb42006dee0 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 0x7fb4201efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 0x7fb4201efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34406/0 (socket says 172.21.15.144:34406) 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 -- 172.21.15.144:0/1953843413 learned_addr learned my addr 172.21.15.144:0/1953843413 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41f7fe700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42006dee0 0x7fb4201f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb424f36700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420071040 0x7fb4200ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 -- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42006dee0 msgr2=0x7fb4201f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42006dee0 0x7fb4201f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 -- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420071040 msgr2=0x7fb4200ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420071040 0x7fb4200ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 -- 172.21.15.144:0/1953843413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100094a0 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41f7fe700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42006dee0 0x7fb4201f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41ffff700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 0x7fb4201efd20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb41000a010 tx=0x7fb410004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410003e50 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb410011630 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42015d900 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4100119c0 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.026+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb42015d620 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.027+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42006e890 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.028+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb410017020 con 0x7fb42006cdc0 2023-11-28T11:51:20.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.028+0000 7fb41cff9700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4000682f0 0x7fb40006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.028+0000 7fb41f7fe700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4000682f0 0x7fb40006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.028+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4100872c0 con 0x7fb42006cdc0 2023-11-28T11:51:20.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.029+0000 7fb41f7fe700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4000682f0 0x7fb40006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4200c6af0 tx=0x7fb408008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.032+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb410053b20 con 0x7fb42006cdc0 2023-11-28T11:51:20.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.256+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4200720a0 con 0x7fb42006cdc0 2023-11-28T11:51:20.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.256+0000 7fb41cff9700 1 -- 172.21.15.144:0/1953843413 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fb410057140 con 0x7fb42006cdc0 2023-11-28T11:51:20.257 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:20.257 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:51:16.866858+0000","modified":"2023-11-28T11:51:19.878262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:51:20.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4000682f0 msgr2=0x7fb40006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4000682f0 0x7fb40006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4200c6af0 tx=0x7fb408008040 comp rx=0 tx=0).stop 2023-11-28T11:51:20.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 msgr2=0x7fb4201efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 0x7fb4201efd20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb41000a010 tx=0x7fb410004750 comp rx=0 tx=0).stop 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 shutdown_connections 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb42006cdc0 0x7fb4201efd20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4000682f0 0x7fb40006a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb420071040 0x7fb4200ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 --2- 172.21.15.144:0/1953843413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb42006dee0 0x7fb4201f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 >> 172.21.15.144:0/1953843413 conn(0x7fb4200d0000 msgr2=0x7fb42006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 shutdown_connections 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.258+0000 7fb426999700 1 -- 172.21.15.144:0/1953843413 wait complete. 2023-11-28T11:51:20.260 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 8 2023-11-28T11:51:20.272 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:20.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.390+0000 7f8274693700 1 Processor -- start 2023-11-28T11:51:20.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.390+0000 7f8274693700 1 -- start start 2023-11-28T11:51:20.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f8274693700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f826c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f8274693700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c071950 con 0x7f826c06dee0 2023-11-28T11:51:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f8274693700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c15d060 con 0x7f826c160540 2023-11-28T11:51:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f8274693700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c15d340 con 0x7f826c071040 2023-11-28T11:51:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f8271c2e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f826c06dee0 0x7f826c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51126/0 (socket says 172.21.15.144:51126) 2023-11-28T11:51:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f8271c2e700 1 -- 172.21.15.144:0/262769441 learned_addr learned my addr 172.21.15.144:0/262769441 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305257667 0 0) 0x7f826c071950 con 0x7f826c06dee0 2023-11-28T11:51:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f827242f700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f826c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.391+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82580035b0 con 0x7f826c06dee0 2023-11-28T11:51:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971958728 0 0) 0x7f82580035b0 con 0x7f826c06dee0 2023-11-28T11:51:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 msgr2=0x7f826c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f826c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 >> v1:172.21.15.177:6789/0 conn(0x7f826c160540 legacy=0x7f826c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c15d620 con 0x7f826c06dee0 2023-11-28T11:51:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f8268001670 con 0x7f826c06dee0 2023-11-28T11:51:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 >> v1:172.21.15.144:6789/0 conn(0x7f826c06dee0 legacy=0x7f826c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258003a30 0x7f8258003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8258004850 0x7f8258004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f82580052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c15d340 con 0x7f8258004850 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825800bbd0 con 0x7f8258003a30 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825800bd90 con 0x7f826c071040 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8258004850 0x7f8258004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827242f700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258003a30 0x7f8258003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8272c30700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f82580052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 -- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 msgr2=0x7f82580052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f82580052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 -- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258003a30 msgr2=0x7f8258003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258003a30 0x7f8258003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 -- 172.21.15.144:0/262769441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580035b0 con 0x7f8258004850 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8272c30700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c071040 0x7f82580052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f8271c2e700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8258004850 0x7f8258004cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f826800c210 tx=0x7f8258005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeb34d24207cc59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268007c10 con 0x7f8258004850 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826800cdd0 con 0x7f8258004850 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.392+0000 7f827142d700 1 -- 172.21.15.144:0/262769441 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268005ba0 con 0x7f8258004850 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8274693700 1 -- 172.21.15.144:0/262769441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8258004850 msgr2=0x7f8258004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8274693700 1 --2- 172.21.15.144:0/262769441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8258004850 0x7f8258004cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f826800c210 tx=0x7f8258005880 comp rx=0 tx=0).stop 2023-11-28T11:51:20.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8272c30700 1 -- 172.21.15.144:0/262769441 reap_dead start 2023-11-28T11:51:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8274693700 1 -- 172.21.15.144:0/262769441 shutdown_connections 2023-11-28T11:51:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8274693700 1 -- 172.21.15.144:0/262769441 >> 172.21.15.144:0/262769441 conn(0x7f826c0d0000 msgr2=0x7f826c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8274693700 1 -- 172.21.15.144:0/262769441 shutdown_connections 2023-11-28T11:51:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.393+0000 7f8274693700 1 -- 172.21.15.144:0/262769441 wait complete. 2023-11-28T11:51:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.394+0000 7f8274693700 1 Processor -- start 2023-11-28T11:51:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.394+0000 7f8274693700 1 -- start start 2023-11-28T11:51:20.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8274693700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258004850 0x7f826c1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8274693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 0x7f826c0c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8274693700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c0cd0b0 0x7f826c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8274693700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0cda60 con 0x7f826c0c4e90 2023-11-28T11:51:20.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8274693700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0c6770 con 0x7f8258004850 2023-11-28T11:51:20.405 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8274693700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c0b3400 con 0x7f826c0cd0b0 2023-11-28T11:51:20.405 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 0x7f826c0c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.405 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 0x7f826c0c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34426/0 (socket says 172.21.15.144:34426) 2023-11-28T11:51:20.405 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 -- 172.21.15.144:0/2806642128 learned_addr learned my addr 172.21.15.144:0/2806642128 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:20.405 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8272c30700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c0cd0b0 0x7f826c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.406 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f827242f700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258004850 0x7f826c1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.406 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 -- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c0cd0b0 msgr2=0x7f826c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c0cd0b0 0x7f826c0cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 -- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258004850 msgr2=0x7f826c1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258004850 0x7f826c1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.395+0000 7f8271c2e700 1 -- 172.21.15.144:0/2806642128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c15ccd0 con 0x7f826c0c4e90 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.396+0000 7f8271c2e700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 0x7f826c0c6230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8268001930 tx=0x7f8268001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.396+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268005ea0 con 0x7f826c0c4e90 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.396+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82680072d0 con 0x7f826c0c4e90 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.396+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c15d900 con 0x7f826c0c4e90 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.396+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268007680 con 0x7f826c0c4e90 2023-11-28T11:51:20.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.396+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c06a270 con 0x7f826c0c4e90 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.397+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c0c6970 con 0x7f826c0c4e90 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.397+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8268010070 con 0x7f826c0c4e90 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.397+0000 7f8262ffd700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82500682f0 0x7f825006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.398+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8268087390 con 0x7f826c0c4e90 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.398+0000 7f827242f700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82500682f0 0x7f825006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.398+0000 7f827242f700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82500682f0 0x7f825006a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f825c009c60 tx=0x7f825c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.408 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.402+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8268053cb0 con 0x7f826c0c4e90 2023-11-28T11:51:20.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.588+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f826c0720a0 con 0x7f826c0c4e90 2023-11-28T11:51:20.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.589+0000 7f8262ffd700 1 -- 172.21.15.144:0/2806642128 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f82680572d0 con 0x7f826c0c4e90 2023-11-28T11:51:20.590 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:20.590 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82500682f0 msgr2=0x7f825006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82500682f0 0x7f825006a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f825c009c60 tx=0x7f825c009210 comp rx=0 tx=0).stop 2023-11-28T11:51:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 msgr2=0x7f826c0c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 0x7f826c0c6230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8268001930 tx=0x7f8268001a00 comp rx=0 tx=0).stop 2023-11-28T11:51:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 shutdown_connections 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82500682f0 0x7f825006a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f826c0cd0b0 0x7f826c0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8258004850 0x7f826c1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 --2- 172.21.15.144:0/2806642128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826c0c4e90 0x7f826c0c6230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 >> 172.21.15.144:0/2806642128 conn(0x7f826c0d0000 msgr2=0x7f826c06a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 shutdown_connections 2023-11-28T11:51:20.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.591+0000 7f8274693700 1 -- 172.21.15.144:0/2806642128 wait complete. 2023-11-28T11:51:20.604 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:51:20.604 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:51:20.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.724+0000 7f62d323d700 1 Processor -- start 2023-11-28T11:51:20.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.724+0000 7f62d323d700 1 -- start start 2023-11-28T11:51:20.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d323d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f62cc154e70 0x7f62cc0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d323d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d323d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc155240 con 0x7f62cc0cf360 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d323d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc15b100 con 0x7f62cc154e70 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d323d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15b3e0 con 0x7f62cc071750 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d0fd9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f62cc154e70 0x7f62cc0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d0fd9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f62cc154e70 0x7f62cc0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36708/0 (socket says 172.21.15.144:36708) 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62cbfff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f62cc0cf360 0x7f62cc071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51638/0 (socket says 172.21.15.144:51638) 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46520/0 (socket says 172.21.15.144:46520) 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d0fd9700 1 -- 172.21.15.144:0/1729688778 learned_addr learned my addr 172.21.15.144:0/1729688778 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 -- 172.21.15.144:0/1729688778 >> v2:172.21.15.177:3300/0 conn(0x7f62cc154e70 msgr2=0x7f62cc0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 --2- 172.21.15.144:0/1729688778 >> v2:172.21.15.177:3300/0 conn(0x7f62cc154e70 0x7f62cc0cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 -- 172.21.15.144:0/1729688778 >> v1:172.21.15.177:6789/0 conn(0x7f62cc0cf360 legacy=0x7f62cc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 -- 172.21.15.144:0/1729688778 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc15b6c0 con 0x7f62cc071750 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.725+0000 7f62d17da700 1 --2- 172.21.15.144:0/1729688778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc15e680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009d90 tx=0x7f62c400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de6985f04a3058f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62cb7fe700 1 -- 172.21.15.144:0/1729688778 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400e070 con 0x7f62cc071750 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62cb7fe700 1 -- 172.21.15.144:0/1729688778 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400bb20 con 0x7f62cc071750 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62cb7fe700 1 -- 172.21.15.144:0/1729688778 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4012430 con 0x7f62cc071750 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 -- 172.21.15.144:0/1729688778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 msgr2=0x7f62cc15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 --2- 172.21.15.144:0/1729688778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc15e680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009d90 tx=0x7f62c400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 -- 172.21.15.144:0/1729688778 shutdown_connections 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 --2- 172.21.15.144:0/1729688778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc15e680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 --2- 172.21.15.144:0/1729688778 >> v2:172.21.15.177:3300/0 conn(0x7f62cc154e70 0x7f62cc0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 -- 172.21.15.144:0/1729688778 >> 172.21.15.144:0/1729688778 conn(0x7f62cc0d0000 msgr2=0x7f62cc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 -- 172.21.15.144:0/1729688778 shutdown_connections 2023-11-28T11:51:20.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.726+0000 7f62d323d700 1 -- 172.21.15.144:0/1729688778 wait complete. 2023-11-28T11:51:20.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.727+0000 7f62d323d700 1 Processor -- start 2023-11-28T11:51:20.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.727+0000 7f62d323d700 1 -- start start 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d323d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d323d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 0x7f62cc0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d323d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62cc154e70 0x7f62cc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d323d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0b2ed0 con 0x7f62cc0cf360 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d323d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0b31b0 con 0x7f62cc154e70 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d323d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0b3490 con 0x7f62cc071750 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 0x7f62cc0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 0x7f62cc0c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34442/0 (socket says 172.21.15.144:34442) 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d0fd9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62cbfff700 1 -- 172.21.15.144:0/4051469842 learned_addr learned my addr 172.21.15.144:0/4051469842 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.728+0000 7f62d17da700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62cc154e70 0x7f62cc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62cbfff700 1 -- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 msgr2=0x7f62cc1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62cbfff700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62cbfff700 1 -- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62cc154e70 msgr2=0x7f62cc0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62cbfff700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62cc154e70 0x7f62cc0c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62cbfff700 1 -- 172.21.15.144:0/4051469842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c4009a70 con 0x7f62cc0cf360 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62d0fd9700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62cbfff700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 0x7f62cc0c7fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c520 tx=0x7f62b800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8005cb0 con 0x7f62cc0cf360 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b8005e10 con 0x7f62cc0cf360 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc0af520 con 0x7f62cc0cf360 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b80072d0 con 0x7f62cc0cf360 2023-11-28T11:51:20.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.729+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc15b9a0 con 0x7f62cc0cf360 2023-11-28T11:51:20.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.730+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc15adb0 con 0x7f62cc0cf360 2023-11-28T11:51:20.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.731+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62b8019030 con 0x7f62cc0cf360 2023-11-28T11:51:20.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.731+0000 7f62c97fa700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f62b00682f0 0x7f62b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:20.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.731+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62b8084df0 con 0x7f62cc0cf360 2023-11-28T11:51:20.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.731+0000 7f62d0fd9700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f62b00682f0 0x7f62b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:20.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.731+0000 7f62d0fd9700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f62b00682f0 0x7f62b006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009f20 tx=0x7f62c00092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:20.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.735+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62b804b5e0 con 0x7f62cc0cf360 2023-11-28T11:51:20.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.915+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62cc0b39b0 con 0x7f62b00682f0 2023-11-28T11:51:20.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.923+0000 7f62c97fa700 1 -- 172.21.15.144:0/4051469842 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286891 (secure 0 0 0) 0x7f62cc0b39b0 con 0x7f62b00682f0 2023-11-28T11:51:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:20.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f62b00682f0 msgr2=0x7f62b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f62b00682f0 0x7f62b006a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009f20 tx=0x7f62c00092c0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 msgr2=0x7f62cc0c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 0x7f62cc0c7fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62b800c520 tx=0x7f62b800c830 comp rx=0 tx=0).stop 2023-11-28T11:51:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 shutdown_connections 2023-11-28T11:51:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f62b00682f0 0x7f62b006a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62cc154e70 0x7f62cc0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62cc071750 0x7f62cc1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 --2- 172.21.15.144:0/4051469842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62cc0cf360 0x7f62cc0c7fc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:20.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 >> 172.21.15.144:0/4051469842 conn(0x7f62cc0d0000 msgr2=0x7f62cc0b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:20.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 shutdown_connections 2023-11-28T11:51:20.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:20.926+0000 7f62d323d700 1 -- 172.21.15.144:0/4051469842 wait complete. 2023-11-28T11:51:20.934 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:51:20.939 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T11:51:20.482227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66016,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908704,"statfs":{"total":773094113280,"available":773026512896,"internally_reserved":0,"allocated":3047424,"data_stored":1845464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021964"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092580+0000","last_change":"2023-11-28T11:51:16.886968+0000","last_active":"2023-11-28T11:51:19.092580+0000","last_peered":"2023-11-28T11:51:19.092580+0000","last_clean":"2023-11-28T11:51:19.092580+0000","last_became_active":"2023-11-28T11:51:16.885939+0000","last_became_peered":"2023-11-28T11:51:16.885939+0000","last_unstale":"2023-11-28T11:51:19.092580+0000","last_undegraded":"2023-11-28T11:51:19.092580+0000","last_fullsized":"2023-11-28T11:51:19.092580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:14.596675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879462+0000","last_change":"2023-11-28T11:51:16.887419+0000","last_active":"2023-11-28T11:51:17.879462+0000","last_peered":"2023-11-28T11:51:17.879462+0000","last_clean":"2023-11-28T11:51:17.879462+0000","last_became_active":"2023-11-28T11:51:16.887120+0000","last_became_peered":"2023-11-28T11:51:16.887120+0000","last_unstale":"2023-11-28T11:51:17.879462+0000","last_undegraded":"2023-11-28T11:51:17.879462+0000","last_fullsized":"2023-11-28T11:51:17.879462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881469+0000","last_change":"2023-11-28T11:51:16.899556+0000","last_active":"2023-11-28T11:51:17.881469+0000","last_peered":"2023-11-28T11:51:17.881469+0000","last_clean":"2023-11-28T11:51:17.881469+0000","last_became_active":"2023-11-28T11:51:16.899088+0000","last_became_peered":"2023-11-28T11:51:16.899088+0000","last_unstale":"2023-11-28T11:51:17.881469+0000","last_undegraded":"2023-11-28T11:51:17.881469+0000","last_fullsized":"2023-11-28T11:51:17.881469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:23.190053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093312+0000","last_change":"2023-11-28T11:51:16.898501+0000","last_active":"2023-11-28T11:51:19.093312+0000","last_peered":"2023-11-28T11:51:19.093312+0000","last_clean":"2023-11-28T11:51:19.093312+0000","last_became_active":"2023-11-28T11:51:16.898263+0000","last_became_peered":"2023-11-28T11:51:16.898263+0000","last_unstale":"2023-11-28T11:51:19.093312+0000","last_undegraded":"2023-11-28T11:51:19.093312+0000","last_fullsized":"2023-11-28T11:51:19.093312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093023+0000","last_change":"2023-11-28T11:51:16.885205+0000","last_active":"2023-11-28T11:51:19.093023+0000","last_peered":"2023-11-28T11:51:19.093023+0000","last_clean":"2023-11-28T11:51:19.093023+0000","last_became_active":"2023-11-28T11:51:16.884964+0000","last_became_peered":"2023-11-28T11:51:16.884964+0000","last_unstale":"2023-11-28T11:51:19.093023+0000","last_undegraded":"2023-11-28T11:51:19.093023+0000","last_fullsized":"2023-11-28T11:51:19.093023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:36.307669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881261+0000","last_change":"2023-11-28T11:51:16.899331+0000","last_active":"2023-11-28T11:51:17.881261+0000","last_peered":"2023-11-28T11:51:17.881261+0000","last_clean":"2023-11-28T11:51:17.881261+0000","last_became_active":"2023-11-28T11:51:16.899026+0000","last_became_peered":"2023-11-28T11:51:16.899026+0000","last_unstale":"2023-11-28T11:51:17.881261+0000","last_undegraded":"2023-11-28T11:51:17.881261+0000","last_fullsized":"2023-11-28T11:51:17.881261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:40.625446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093069+0000","last_change":"2023-11-28T11:51:16.895279+0000","last_active":"2023-11-28T11:51:19.093069+0000","last_peered":"2023-11-28T11:51:19.093069+0000","last_clean":"2023-11-28T11:51:19.093069+0000","last_became_active":"2023-11-28T11:51:16.894949+0000","last_became_peered":"2023-11-28T11:51:16.894949+0000","last_unstale":"2023-11-28T11:51:19.093069+0000","last_undegraded":"2023-11-28T11:51:19.093069+0000","last_fullsized":"2023-11-28T11:51:19.093069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.882353+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.882010+0000","last_became_peered":"2023-11-28T11:51:16.882010+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093331+0000","last_change":"2023-11-28T11:51:16.885987+0000","last_active":"2023-11-28T11:51:19.093331+0000","last_peered":"2023-11-28T11:51:19.093331+0000","last_clean":"2023-11-28T11:51:19.093331+0000","last_became_active":"2023-11-28T11:51:16.885744+0000","last_became_peered":"2023-11-28T11:51:16.885744+0000","last_unstale":"2023-11-28T11:51:19.093331+0000","last_undegraded":"2023-11-28T11:51:19.093331+0000","last_fullsized":"2023-11-28T11:51:19.093331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:20:59.112675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879642+0000","last_change":"2023-11-28T11:51:16.898873+0000","last_active":"2023-11-28T11:51:17.879642+0000","last_peered":"2023-11-28T11:51:17.879642+0000","last_clean":"2023-11-28T11:51:17.879642+0000","last_became_active":"2023-11-28T11:51:16.898567+0000","last_became_peered":"2023-11-28T11:51:16.898567+0000","last_unstale":"2023-11-28T11:51:17.879642+0000","last_undegraded":"2023-11-28T11:51:17.879642+0000","last_fullsized":"2023-11-28T11:51:17.879642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881611+0000","last_change":"2023-11-28T11:51:16.892348+0000","last_active":"2023-11-28T11:51:17.881611+0000","last_peered":"2023-11-28T11:51:17.881611+0000","last_clean":"2023-11-28T11:51:17.881611+0000","last_became_active":"2023-11-28T11:51:16.891451+0000","last_became_peered":"2023-11-28T11:51:16.891451+0000","last_unstale":"2023-11-28T11:51:17.881611+0000","last_undegraded":"2023-11-28T11:51:17.881611+0000","last_fullsized":"2023-11-28T11:51:17.881611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:37.035204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092706+0000","last_change":"2023-11-28T11:51:16.893555+0000","last_active":"2023-11-28T11:51:19.092706+0000","last_peered":"2023-11-28T11:51:19.092706+0000","last_clean":"2023-11-28T11:51:19.092706+0000","last_became_active":"2023-11-28T11:51:16.893296+0000","last_became_peered":"2023-11-28T11:51:16.893296+0000","last_unstale":"2023-11-28T11:51:19.092706+0000","last_undegraded":"2023-11-28T11:51:19.092706+0000","last_fullsized":"2023-11-28T11:51:19.092706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:13.298585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092713+0000","last_change":"2023-11-28T11:51:16.885357+0000","last_active":"2023-11-28T11:51:19.092713+0000","last_peered":"2023-11-28T11:51:19.092713+0000","last_clean":"2023-11-28T11:51:19.092713+0000","last_became_active":"2023-11-28T11:51:16.885098+0000","last_became_peered":"2023-11-28T11:51:16.885098+0000","last_unstale":"2023-11-28T11:51:19.092713+0000","last_undegraded":"2023-11-28T11:51:19.092713+0000","last_fullsized":"2023-11-28T11:51:19.092713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:18.850790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880260+0000","last_change":"2023-11-28T11:51:16.882443+0000","last_active":"2023-11-28T11:51:17.880260+0000","last_peered":"2023-11-28T11:51:17.880260+0000","last_clean":"2023-11-28T11:51:17.880260+0000","last_became_active":"2023-11-28T11:51:16.882164+0000","last_became_peered":"2023-11-28T11:51:16.882164+0000","last_unstale":"2023-11-28T11:51:17.880260+0000","last_undegraded":"2023-11-28T11:51:17.880260+0000","last_fullsized":"2023-11-28T11:51:17.880260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093567+0000","last_change":"2023-11-28T11:51:16.895797+0000","last_active":"2023-11-28T11:51:19.093567+0000","last_peered":"2023-11-28T11:51:19.093567+0000","last_clean":"2023-11-28T11:51:19.093567+0000","last_became_active":"2023-11-28T11:51:16.895517+0000","last_became_peered":"2023-11-28T11:51:16.895517+0000","last_unstale":"2023-11-28T11:51:19.093567+0000","last_undegraded":"2023-11-28T11:51:19.093567+0000","last_fullsized":"2023-11-28T11:51:19.093567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881087+0000","last_change":"2023-11-28T11:51:16.899351+0000","last_active":"2023-11-28T11:51:17.881087+0000","last_peered":"2023-11-28T11:51:17.881087+0000","last_clean":"2023-11-28T11:51:17.881087+0000","last_became_active":"2023-11-28T11:51:16.899044+0000","last_became_peered":"2023-11-28T11:51:16.899044+0000","last_unstale":"2023-11-28T11:51:17.881087+0000","last_undegraded":"2023-11-28T11:51:17.881087+0000","last_fullsized":"2023-11-28T11:51:17.881087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:09.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093414+0000","last_change":"2023-11-28T11:51:16.885961+0000","last_active":"2023-11-28T11:51:19.093414+0000","last_peered":"2023-11-28T11:51:19.093414+0000","last_clean":"2023-11-28T11:51:19.093414+0000","last_became_active":"2023-11-28T11:51:16.885709+0000","last_became_peered":"2023-11-28T11:51:16.885709+0000","last_unstale":"2023-11-28T11:51:19.093414+0000","last_undegraded":"2023-11-28T11:51:19.093414+0000","last_fullsized":"2023-11-28T11:51:19.093414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:13.291091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881602+0000","last_change":"2023-11-28T11:51:16.893897+0000","last_active":"2023-11-28T11:51:17.881602+0000","last_peered":"2023-11-28T11:51:17.881602+0000","last_clean":"2023-11-28T11:51:17.881602+0000","last_became_active":"2023-11-28T11:51:16.893541+0000","last_became_peered":"2023-11-28T11:51:16.893541+0000","last_unstale":"2023-11-28T11:51:17.881602+0000","last_undegraded":"2023-11-28T11:51:17.881602+0000","last_fullsized":"2023-11-28T11:51:17.881602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092888+0000","last_change":"2023-11-28T11:51:16.887327+0000","last_active":"2023-11-28T11:51:19.092888+0000","last_peered":"2023-11-28T11:51:19.092888+0000","last_clean":"2023-11-28T11:51:19.092888+0000","last_became_active":"2023-11-28T11:51:16.887062+0000","last_became_peered":"2023-11-28T11:51:16.887062+0000","last_unstale":"2023-11-28T11:51:19.092888+0000","last_undegraded":"2023-11-28T11:51:19.092888+0000","last_fullsized":"2023-11-28T11:51:19.092888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:49.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881480+0000","last_change":"2023-11-28T11:51:16.889122+0000","last_active":"2023-11-28T11:51:17.881480+0000","last_peered":"2023-11-28T11:51:17.881480+0000","last_clean":"2023-11-28T11:51:17.881480+0000","last_became_active":"2023-11-28T11:51:16.888849+0000","last_became_peered":"2023-11-28T11:51:16.888849+0000","last_unstale":"2023-11-28T11:51:17.881480+0000","last_undegraded":"2023-11-28T11:51:17.881480+0000","last_fullsized":"2023-11-28T11:51:17.881480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093486+0000","last_change":"2023-11-28T11:51:16.895113+0000","last_active":"2023-11-28T11:51:19.093486+0000","last_peered":"2023-11-28T11:51:19.093486+0000","last_clean":"2023-11-28T11:51:19.093486+0000","last_became_active":"2023-11-28T11:51:16.894793+0000","last_became_peered":"2023-11-28T11:51:16.894793+0000","last_unstale":"2023-11-28T11:51:19.093486+0000","last_undegraded":"2023-11-28T11:51:19.093486+0000","last_fullsized":"2023-11-28T11:51:19.093486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898452+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897946+0000","last_became_peered":"2023-11-28T11:51:16.897946+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:20.222278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880183+0000","last_change":"2023-11-28T11:51:16.898903+0000","last_active":"2023-11-28T11:51:17.880183+0000","last_peered":"2023-11-28T11:51:17.880183+0000","last_clean":"2023-11-28T11:51:17.880183+0000","last_became_active":"2023-11-28T11:51:16.898719+0000","last_became_peered":"2023-11-28T11:51:16.898719+0000","last_unstale":"2023-11-28T11:51:17.880183+0000","last_undegraded":"2023-11-28T11:51:17.880183+0000","last_fullsized":"2023-11-28T11:51:17.880183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093156+0000","last_change":"2023-11-28T11:51:16.895329+0000","last_active":"2023-11-28T11:51:19.093156+0000","last_peered":"2023-11-28T11:51:19.093156+0000","last_clean":"2023-11-28T11:51:19.093156+0000","last_became_active":"2023-11-28T11:51:16.894998+0000","last_became_peered":"2023-11-28T11:51:16.894998+0000","last_unstale":"2023-11-28T11:51:19.093156+0000","last_undegraded":"2023-11-28T11:51:19.093156+0000","last_fullsized":"2023-11-28T11:51:19.093156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093690+0000","last_change":"2023-11-28T11:51:16.889063+0000","last_active":"2023-11-28T11:51:19.093690+0000","last_peered":"2023-11-28T11:51:19.093690+0000","last_clean":"2023-11-28T11:51:19.093690+0000","last_became_active":"2023-11-28T11:51:16.888711+0000","last_became_peered":"2023-11-28T11:51:16.888711+0000","last_unstale":"2023-11-28T11:51:19.093690+0000","last_undegraded":"2023-11-28T11:51:19.093690+0000","last_fullsized":"2023-11-28T11:51:19.093690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093545+0000","last_change":"2023-11-28T11:51:16.882290+0000","last_active":"2023-11-28T11:51:19.093545+0000","last_peered":"2023-11-28T11:51:19.093545+0000","last_clean":"2023-11-28T11:51:19.093545+0000","last_became_active":"2023-11-28T11:51:16.882049+0000","last_became_peered":"2023-11-28T11:51:16.882049+0000","last_unstale":"2023-11-28T11:51:19.093545+0000","last_undegraded":"2023-11-28T11:51:19.093545+0000","last_fullsized":"2023-11-28T11:51:19.093545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879435+0000","last_change":"2023-11-28T11:51:16.896720+0000","last_active":"2023-11-28T11:51:17.879435+0000","last_peered":"2023-11-28T11:51:17.879435+0000","last_clean":"2023-11-28T11:51:17.879435+0000","last_became_active":"2023-11-28T11:51:16.896429+0000","last_became_peered":"2023-11-28T11:51:16.896429+0000","last_unstale":"2023-11-28T11:51:17.879435+0000","last_undegraded":"2023-11-28T11:51:17.879435+0000","last_fullsized":"2023-11-28T11:51:17.879435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881315+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:17.881315+0000","last_peered":"2023-11-28T11:51:17.881315+0000","last_clean":"2023-11-28T11:51:17.881315+0000","last_became_active":"2023-11-28T11:51:16.482973+0000","last_became_peered":"2023-11-28T11:51:16.482973+0000","last_unstale":"2023-11-28T11:51:17.881315+0000","last_undegraded":"2023-11-28T11:51:17.881315+0000","last_fullsized":"2023-11-28T11:51:17.881315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880643+0000","last_change":"2023-11-28T11:51:16.887525+0000","last_active":"2023-11-28T11:51:17.880643+0000","last_peered":"2023-11-28T11:51:17.880643+0000","last_clean":"2023-11-28T11:51:17.880643+0000","last_became_active":"2023-11-28T11:51:16.887247+0000","last_became_peered":"2023-11-28T11:51:16.887247+0000","last_unstale":"2023-11-28T11:51:17.880643+0000","last_undegraded":"2023-11-28T11:51:17.880643+0000","last_fullsized":"2023-11-28T11:51:17.880643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093032+0000","last_change":"2023-11-28T11:51:16.481846+0000","last_active":"2023-11-28T11:51:19.093032+0000","last_peered":"2023-11-28T11:51:19.093032+0000","last_clean":"2023-11-28T11:51:19.093032+0000","last_became_active":"2023-11-28T11:51:16.481653+0000","last_became_peered":"2023-11-28T11:51:16.481653+0000","last_unstale":"2023-11-28T11:51:19.093032+0000","last_undegraded":"2023-11-28T11:51:19.093032+0000","last_fullsized":"2023-11-28T11:51:19.093032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:21.555920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093520+0000","last_change":"2023-11-28T11:51:16.895323+0000","last_active":"2023-11-28T11:51:19.093520+0000","last_peered":"2023-11-28T11:51:19.093520+0000","last_clean":"2023-11-28T11:51:19.093520+0000","last_became_active":"2023-11-28T11:51:16.895030+0000","last_became_peered":"2023-11-28T11:51:16.895030+0000","last_unstale":"2023-11-28T11:51:19.093520+0000","last_undegraded":"2023-11-28T11:51:19.093520+0000","last_fullsized":"2023-11-28T11:51:19.093520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:59.340661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880302+0000","last_change":"2023-11-28T11:51:16.488974+0000","last_active":"2023-11-28T11:51:17.880302+0000","last_peered":"2023-11-28T11:51:17.880302+0000","last_clean":"2023-11-28T11:51:17.880302+0000","last_became_active":"2023-11-28T11:51:16.488697+0000","last_became_peered":"2023-11-28T11:51:16.488697+0000","last_unstale":"2023-11-28T11:51:17.880302+0000","last_undegraded":"2023-11-28T11:51:17.880302+0000","last_fullsized":"2023-11-28T11:51:17.880302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093841+0000","last_change":"2023-11-28T11:51:16.888129+0000","last_active":"2023-11-28T11:51:19.093841+0000","last_peered":"2023-11-28T11:51:19.093841+0000","last_clean":"2023-11-28T11:51:19.093841+0000","last_became_active":"2023-11-28T11:51:16.887848+0000","last_became_peered":"2023-11-28T11:51:16.887848+0000","last_unstale":"2023-11-28T11:51:19.093841+0000","last_undegraded":"2023-11-28T11:51:19.093841+0000","last_fullsized":"2023-11-28T11:51:19.093841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:42.239688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093488+0000","last_change":"2023-11-28T11:51:16.488283+0000","last_active":"2023-11-28T11:51:19.093488+0000","last_peered":"2023-11-28T11:51:19.093488+0000","last_clean":"2023-11-28T11:51:19.093488+0000","last_became_active":"2023-11-28T11:51:16.487994+0000","last_became_peered":"2023-11-28T11:51:16.487994+0000","last_unstale":"2023-11-28T11:51:19.093488+0000","last_undegraded":"2023-11-28T11:51:19.093488+0000","last_fullsized":"2023-11-28T11:51:19.093488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:04.545703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880313+0000","last_change":"2023-11-28T11:51:16.891442+0000","last_active":"2023-11-28T11:51:17.880313+0000","last_peered":"2023-11-28T11:51:17.880313+0000","last_clean":"2023-11-28T11:51:17.880313+0000","last_became_active":"2023-11-28T11:51:16.891180+0000","last_became_peered":"2023-11-28T11:51:16.891180+0000","last_unstale":"2023-11-28T11:51:17.880313+0000","last_undegraded":"2023-11-28T11:51:17.880313+0000","last_fullsized":"2023-11-28T11:51:17.880313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879563+0000","last_change":"2023-11-28T11:51:16.484401+0000","last_active":"2023-11-28T11:51:17.879563+0000","last_peered":"2023-11-28T11:51:17.879563+0000","last_clean":"2023-11-28T11:51:17.879563+0000","last_became_active":"2023-11-28T11:51:16.484103+0000","last_became_peered":"2023-11-28T11:51:16.484103+0000","last_unstale":"2023-11-28T11:51:17.879563+0000","last_undegraded":"2023-11-28T11:51:17.879563+0000","last_fullsized":"2023-11-28T11:51:17.879563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880312+0000","last_change":"2023-11-28T11:51:16.898526+0000","last_active":"2023-11-28T11:51:17.880312+0000","last_peered":"2023-11-28T11:51:17.880312+0000","last_clean":"2023-11-28T11:51:17.880312+0000","last_became_active":"2023-11-28T11:51:16.897658+0000","last_became_peered":"2023-11-28T11:51:16.897658+0000","last_unstale":"2023-11-28T11:51:17.880312+0000","last_undegraded":"2023-11-28T11:51:17.880312+0000","last_fullsized":"2023-11-28T11:51:17.880312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093666+0000","last_change":"2023-11-28T11:51:16.484674+0000","last_active":"2023-11-28T11:51:19.093666+0000","last_peered":"2023-11-28T11:51:19.093666+0000","last_clean":"2023-11-28T11:51:19.093666+0000","last_became_active":"2023-11-28T11:51:16.484350+0000","last_became_peered":"2023-11-28T11:51:16.484350+0000","last_unstale":"2023-11-28T11:51:19.093666+0000","last_undegraded":"2023-11-28T11:51:19.093666+0000","last_fullsized":"2023-11-28T11:51:19.093666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:13.351037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093718+0000","last_change":"2023-11-28T11:51:16.889523+0000","last_active":"2023-11-28T11:51:19.093718+0000","last_peered":"2023-11-28T11:51:19.093718+0000","last_clean":"2023-11-28T11:51:19.093718+0000","last_became_active":"2023-11-28T11:51:16.889323+0000","last_became_peered":"2023-11-28T11:51:16.889323+0000","last_unstale":"2023-11-28T11:51:19.093718+0000","last_undegraded":"2023-11-28T11:51:19.093718+0000","last_fullsized":"2023-11-28T11:51:19.093718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881658+0000","last_change":"2023-11-28T11:51:16.482345+0000","last_active":"2023-11-28T11:51:17.881658+0000","last_peered":"2023-11-28T11:51:17.881658+0000","last_clean":"2023-11-28T11:51:17.881658+0000","last_became_active":"2023-11-28T11:51:16.482106+0000","last_became_peered":"2023-11-28T11:51:16.482106+0000","last_unstale":"2023-11-28T11:51:17.881658+0000","last_undegraded":"2023-11-28T11:51:17.881658+0000","last_fullsized":"2023-11-28T11:51:17.881658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881274+0000","last_change":"2023-11-28T11:51:16.890129+0000","last_active":"2023-11-28T11:51:17.881274+0000","last_peered":"2023-11-28T11:51:17.881274+0000","last_clean":"2023-11-28T11:51:17.881274+0000","last_became_active":"2023-11-28T11:51:16.889849+0000","last_became_peered":"2023-11-28T11:51:16.889849+0000","last_unstale":"2023-11-28T11:51:17.881274+0000","last_undegraded":"2023-11-28T11:51:17.881274+0000","last_fullsized":"2023-11-28T11:51:17.881274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093375+0000","last_change":"2023-11-28T11:51:16.485194+0000","last_active":"2023-11-28T11:51:19.093375+0000","last_peered":"2023-11-28T11:51:19.093375+0000","last_clean":"2023-11-28T11:51:19.093375+0000","last_became_active":"2023-11-28T11:51:16.484873+0000","last_became_peered":"2023-11-28T11:51:16.484873+0000","last_unstale":"2023-11-28T11:51:19.093375+0000","last_undegraded":"2023-11-28T11:51:19.093375+0000","last_fullsized":"2023-11-28T11:51:19.093375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880521+0000","last_change":"2023-11-28T11:51:16.887321+0000","last_active":"2023-11-28T11:51:17.880521+0000","last_peered":"2023-11-28T11:51:17.880521+0000","last_clean":"2023-11-28T11:51:17.880521+0000","last_became_active":"2023-11-28T11:51:16.886989+0000","last_became_peered":"2023-11-28T11:51:16.886989+0000","last_unstale":"2023-11-28T11:51:17.880521+0000","last_undegraded":"2023-11-28T11:51:17.880521+0000","last_fullsized":"2023-11-28T11:51:17.880521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093431+0000","last_change":"2023-11-28T11:51:16.486311+0000","last_active":"2023-11-28T11:51:19.093431+0000","last_peered":"2023-11-28T11:51:19.093431+0000","last_clean":"2023-11-28T11:51:19.093431+0000","last_became_active":"2023-11-28T11:51:16.485952+0000","last_became_peered":"2023-11-28T11:51:16.485952+0000","last_unstale":"2023-11-28T11:51:19.093431+0000","last_undegraded":"2023-11-28T11:51:19.093431+0000","last_fullsized":"2023-11-28T11:51:19.093431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:13.615057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879568+0000","last_change":"2023-11-28T11:51:16.896713+0000","last_active":"2023-11-28T11:51:17.879568+0000","last_peered":"2023-11-28T11:51:17.879568+0000","last_clean":"2023-11-28T11:51:17.879568+0000","last_became_active":"2023-11-28T11:51:16.896490+0000","last_became_peered":"2023-11-28T11:51:16.896490+0000","last_unstale":"2023-11-28T11:51:17.879568+0000","last_undegraded":"2023-11-28T11:51:17.879568+0000","last_fullsized":"2023-11-28T11:51:17.879568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093447+0000","last_change":"2023-11-28T11:51:16.485718+0000","last_active":"2023-11-28T11:51:19.093447+0000","last_peered":"2023-11-28T11:51:19.093447+0000","last_clean":"2023-11-28T11:51:19.093447+0000","last_became_active":"2023-11-28T11:51:16.485279+0000","last_became_peered":"2023-11-28T11:51:16.485279+0000","last_unstale":"2023-11-28T11:51:19.093447+0000","last_undegraded":"2023-11-28T11:51:19.093447+0000","last_fullsized":"2023-11-28T11:51:19.093447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898233+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897971+0000","last_became_peered":"2023-11-28T11:51:16.897971+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:59.713494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092990+0000","last_change":"2023-11-28T11:51:16.489032+0000","last_active":"2023-11-28T11:51:19.092990+0000","last_peered":"2023-11-28T11:51:19.092990+0000","last_clean":"2023-11-28T11:51:19.092990+0000","last_became_active":"2023-11-28T11:51:16.488752+0000","last_became_peered":"2023-11-28T11:51:16.488752+0000","last_unstale":"2023-11-28T11:51:19.092990+0000","last_undegraded":"2023-11-28T11:51:19.092990+0000","last_fullsized":"2023-11-28T11:51:19.092990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880771+0000","last_change":"2023-11-28T11:51:16.897679+0000","last_active":"2023-11-28T11:51:17.880771+0000","last_peered":"2023-11-28T11:51:17.880771+0000","last_clean":"2023-11-28T11:51:17.880771+0000","last_became_active":"2023-11-28T11:51:16.897336+0000","last_became_peered":"2023-11-28T11:51:16.897336+0000","last_unstale":"2023-11-28T11:51:17.880771+0000","last_undegraded":"2023-11-28T11:51:17.880771+0000","last_fullsized":"2023-11-28T11:51:17.880771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093751+0000","last_change":"2023-11-28T11:51:16.485545+0000","last_active":"2023-11-28T11:51:19.093751+0000","last_peered":"2023-11-28T11:51:19.093751+0000","last_clean":"2023-11-28T11:51:19.093751+0000","last_became_active":"2023-11-28T11:51:16.485097+0000","last_became_peered":"2023-11-28T11:51:16.485097+0000","last_unstale":"2023-11-28T11:51:19.093751+0000","last_undegraded":"2023-11-28T11:51:19.093751+0000","last_fullsized":"2023-11-28T11:51:19.093751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:41.232205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.896139+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.895569+0000","last_became_peered":"2023-11-28T11:51:16.895569+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093685+0000","last_change":"2023-11-28T11:51:16.485744+0000","last_active":"2023-11-28T11:51:19.093685+0000","last_peered":"2023-11-28T11:51:19.093685+0000","last_clean":"2023-11-28T11:51:19.093685+0000","last_became_active":"2023-11-28T11:51:16.485468+0000","last_became_peered":"2023-11-28T11:51:16.485468+0000","last_unstale":"2023-11-28T11:51:19.093685+0000","last_undegraded":"2023-11-28T11:51:19.093685+0000","last_fullsized":"2023-11-28T11:51:19.093685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881144+0000","last_change":"2023-11-28T11:51:16.887450+0000","last_active":"2023-11-28T11:51:17.881144+0000","last_peered":"2023-11-28T11:51:17.881144+0000","last_clean":"2023-11-28T11:51:17.881144+0000","last_became_active":"2023-11-28T11:51:16.887190+0000","last_became_peered":"2023-11-28T11:51:16.887190+0000","last_unstale":"2023-11-28T11:51:17.881144+0000","last_undegraded":"2023-11-28T11:51:17.881144+0000","last_fullsized":"2023-11-28T11:51:17.881144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:00.113811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093607+0000","last_change":"2023-11-28T11:51:16.485512+0000","last_active":"2023-11-28T11:51:19.093607+0000","last_peered":"2023-11-28T11:51:19.093607+0000","last_clean":"2023-11-28T11:51:19.093607+0000","last_became_active":"2023-11-28T11:51:16.485186+0000","last_became_peered":"2023-11-28T11:51:16.485186+0000","last_unstale":"2023-11-28T11:51:19.093607+0000","last_undegraded":"2023-11-28T11:51:19.093607+0000","last_fullsized":"2023-11-28T11:51:19.093607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:41.955746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093236+0000","last_change":"2023-11-28T11:51:16.888771+0000","last_active":"2023-11-28T11:51:19.093236+0000","last_peered":"2023-11-28T11:51:19.093236+0000","last_clean":"2023-11-28T11:51:19.093236+0000","last_became_active":"2023-11-28T11:51:16.888179+0000","last_became_peered":"2023-11-28T11:51:16.888179+0000","last_unstale":"2023-11-28T11:51:19.093236+0000","last_undegraded":"2023-11-28T11:51:19.093236+0000","last_fullsized":"2023-11-28T11:51:19.093236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879495+0000","last_change":"2023-11-28T11:51:16.475494+0000","last_active":"2023-11-28T11:51:17.879495+0000","last_peered":"2023-11-28T11:51:17.879495+0000","last_clean":"2023-11-28T11:51:17.879495+0000","last_became_active":"2023-11-28T11:51:16.475189+0000","last_became_peered":"2023-11-28T11:51:16.475189+0000","last_unstale":"2023-11-28T11:51:17.879495+0000","last_undegraded":"2023-11-28T11:51:17.879495+0000","last_fullsized":"2023-11-28T11:51:17.879495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092876+0000","last_change":"2023-11-28T11:51:16.889569+0000","last_active":"2023-11-28T11:51:19.092876+0000","last_peered":"2023-11-28T11:51:19.092876+0000","last_clean":"2023-11-28T11:51:19.092876+0000","last_became_active":"2023-11-28T11:51:16.888349+0000","last_became_peered":"2023-11-28T11:51:16.888349+0000","last_unstale":"2023-11-28T11:51:19.092876+0000","last_undegraded":"2023-11-28T11:51:19.092876+0000","last_fullsized":"2023-11-28T11:51:19.092876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:37.975550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880421+0000","last_change":"2023-11-28T11:51:16.491156+0000","last_active":"2023-11-28T11:51:17.880421+0000","last_peered":"2023-11-28T11:51:17.880421+0000","last_clean":"2023-11-28T11:51:17.880421+0000","last_became_active":"2023-11-28T11:51:16.490844+0000","last_became_peered":"2023-11-28T11:51:16.490844+0000","last_unstale":"2023-11-28T11:51:17.880421+0000","last_undegraded":"2023-11-28T11:51:17.880421+0000","last_fullsized":"2023-11-28T11:51:17.880421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881598+0000","last_change":"2023-11-28T11:51:16.890271+0000","last_active":"2023-11-28T11:51:17.881598+0000","last_peered":"2023-11-28T11:51:17.881598+0000","last_clean":"2023-11-28T11:51:17.881598+0000","last_became_active":"2023-11-28T11:51:16.889985+0000","last_became_peered":"2023-11-28T11:51:16.889985+0000","last_unstale":"2023-11-28T11:51:17.881598+0000","last_undegraded":"2023-11-28T11:51:17.881598+0000","last_fullsized":"2023-11-28T11:51:17.881598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:03.275473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093142+0000","last_change":"2023-11-28T11:51:16.482742+0000","last_active":"2023-11-28T11:51:19.093142+0000","last_peered":"2023-11-28T11:51:19.093142+0000","last_clean":"2023-11-28T11:51:19.093142+0000","last_became_active":"2023-11-28T11:51:16.482411+0000","last_became_peered":"2023-11-28T11:51:16.482411+0000","last_unstale":"2023-11-28T11:51:19.093142+0000","last_undegraded":"2023-11-28T11:51:19.093142+0000","last_fullsized":"2023-11-28T11:51:19.093142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881716+0000","last_change":"2023-11-28T11:51:16.893619+0000","last_active":"2023-11-28T11:51:17.881716+0000","last_peered":"2023-11-28T11:51:17.881716+0000","last_clean":"2023-11-28T11:51:17.881716+0000","last_became_active":"2023-11-28T11:51:16.893353+0000","last_became_peered":"2023-11-28T11:51:16.893353+0000","last_unstale":"2023-11-28T11:51:17.881716+0000","last_undegraded":"2023-11-28T11:51:17.881716+0000","last_fullsized":"2023-11-28T11:51:17.881716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881245+0000","last_change":"2023-11-28T11:51:16.480268+0000","last_active":"2023-11-28T11:51:17.881245+0000","last_peered":"2023-11-28T11:51:17.881245+0000","last_clean":"2023-11-28T11:51:17.881245+0000","last_became_active":"2023-11-28T11:51:16.479377+0000","last_became_peered":"2023-11-28T11:51:16.479377+0000","last_unstale":"2023-11-28T11:51:17.881245+0000","last_undegraded":"2023-11-28T11:51:17.881245+0000","last_fullsized":"2023-11-28T11:51:17.881245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881631+0000","last_change":"2023-11-28T11:51:16.890147+0000","last_active":"2023-11-28T11:51:17.881631+0000","last_peered":"2023-11-28T11:51:17.881631+0000","last_clean":"2023-11-28T11:51:17.881631+0000","last_became_active":"2023-11-28T11:51:16.889833+0000","last_became_peered":"2023-11-28T11:51:16.889833+0000","last_unstale":"2023-11-28T11:51:17.881631+0000","last_undegraded":"2023-11-28T11:51:17.881631+0000","last_fullsized":"2023-11-28T11:51:17.881631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880209+0000","last_change":"2023-11-28T11:51:16.478507+0000","last_active":"2023-11-28T11:51:17.880209+0000","last_peered":"2023-11-28T11:51:17.880209+0000","last_clean":"2023-11-28T11:51:17.880209+0000","last_became_active":"2023-11-28T11:51:16.478034+0000","last_became_peered":"2023-11-28T11:51:16.478034+0000","last_unstale":"2023-11-28T11:51:17.880209+0000","last_undegraded":"2023-11-28T11:51:17.880209+0000","last_fullsized":"2023-11-28T11:51:17.880209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093877+0000","last_change":"2023-11-28T11:51:16.885456+0000","last_active":"2023-11-28T11:51:19.093877+0000","last_peered":"2023-11-28T11:51:19.093877+0000","last_clean":"2023-11-28T11:51:19.093877+0000","last_became_active":"2023-11-28T11:51:16.885208+0000","last_became_peered":"2023-11-28T11:51:16.885208+0000","last_unstale":"2023-11-28T11:51:19.093877+0000","last_undegraded":"2023-11-28T11:51:19.093877+0000","last_fullsized":"2023-11-28T11:51:19.093877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:24.014435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880712+0000","last_change":"2023-11-28T11:51:16.483690+0000","last_active":"2023-11-28T11:51:17.880712+0000","last_peered":"2023-11-28T11:51:17.880712+0000","last_clean":"2023-11-28T11:51:17.880712+0000","last_became_active":"2023-11-28T11:51:16.483351+0000","last_became_peered":"2023-11-28T11:51:16.483351+0000","last_unstale":"2023-11-28T11:51:17.880712+0000","last_undegraded":"2023-11-28T11:51:17.880712+0000","last_fullsized":"2023-11-28T11:51:17.880712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877898+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:51:16.877898+0000","last_peered":"2023-11-28T11:51:16.877898+0000","last_clean":"2023-11-28T11:51:16.877898+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:51:16.877898+0000","last_undegraded":"2023-11-28T11:51:16.877898+0000","last_fullsized":"2023-11-28T11:51:16.877898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880121+0000","last_change":"2023-11-28T11:51:16.897901+0000","last_active":"2023-11-28T11:51:17.880121+0000","last_peered":"2023-11-28T11:51:17.880121+0000","last_clean":"2023-11-28T11:51:17.880121+0000","last_became_active":"2023-11-28T11:51:16.897621+0000","last_became_peered":"2023-11-28T11:51:16.897621+0000","last_unstale":"2023-11-28T11:51:17.880121+0000","last_undegraded":"2023-11-28T11:51:17.880121+0000","last_fullsized":"2023-11-28T11:51:17.880121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876559+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:51:16.876559+0000","last_peered":"2023-11-28T11:51:16.876559+0000","last_clean":"2023-11-28T11:51:16.876559+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:51:16.876559+0000","last_undegraded":"2023-11-28T11:51:16.876559+0000","last_fullsized":"2023-11-28T11:51:16.876559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093592+0000","last_change":"2023-11-28T11:51:16.486571+0000","last_active":"2023-11-28T11:51:19.093592+0000","last_peered":"2023-11-28T11:51:19.093592+0000","last_clean":"2023-11-28T11:51:19.093592+0000","last_became_active":"2023-11-28T11:51:16.486338+0000","last_became_peered":"2023-11-28T11:51:16.486338+0000","last_unstale":"2023-11-28T11:51:19.093592+0000","last_undegraded":"2023-11-28T11:51:19.093592+0000","last_fullsized":"2023-11-28T11:51:19.093592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:54.333661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881136+0000","last_change":"2023-11-28T11:51:16.889991+0000","last_active":"2023-11-28T11:51:17.881136+0000","last_peered":"2023-11-28T11:51:17.881136+0000","last_clean":"2023-11-28T11:51:17.881136+0000","last_became_active":"2023-11-28T11:51:16.889694+0000","last_became_peered":"2023-11-28T11:51:16.889694+0000","last_unstale":"2023-11-28T11:51:17.881136+0000","last_undegraded":"2023-11-28T11:51:17.881136+0000","last_fullsized":"2023-11-28T11:51:17.881136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.870699+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:51:16.870699+0000","last_peered":"2023-11-28T11:51:16.870699+0000","last_clean":"2023-11-28T11:51:16.870699+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:51:16.870699+0000","last_undegraded":"2023-11-28T11:51:16.870699+0000","last_fullsized":"2023-11-28T11:51:16.870699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877332+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:51:16.877332+0000","last_peered":"2023-11-28T11:51:16.877332+0000","last_clean":"2023-11-28T11:51:16.877332+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:51:16.877332+0000","last_undegraded":"2023-11-28T11:51:16.877332+0000","last_fullsized":"2023-11-28T11:51:16.877332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093112+0000","last_change":"2023-11-28T11:51:16.486764+0000","last_active":"2023-11-28T11:51:19.093112+0000","last_peered":"2023-11-28T11:51:19.093112+0000","last_clean":"2023-11-28T11:51:19.093112+0000","last_became_active":"2023-11-28T11:51:16.486448+0000","last_became_peered":"2023-11-28T11:51:16.486448+0000","last_unstale":"2023-11-28T11:51:19.093112+0000","last_undegraded":"2023-11-28T11:51:19.093112+0000","last_fullsized":"2023-11-28T11:51:19.093112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:15.029206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093385+0000","last_change":"2023-11-28T11:51:16.888008+0000","last_active":"2023-11-28T11:51:19.093385+0000","last_peered":"2023-11-28T11:51:19.093385+0000","last_clean":"2023-11-28T11:51:19.093385+0000","last_became_active":"2023-11-28T11:51:16.887745+0000","last_became_peered":"2023-11-28T11:51:16.887745+0000","last_unstale":"2023-11-28T11:51:19.093385+0000","last_undegraded":"2023-11-28T11:51:19.093385+0000","last_fullsized":"2023-11-28T11:51:19.093385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:39.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876792+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:51:16.876792+0000","last_peered":"2023-11-28T11:51:16.876792+0000","last_clean":"2023-11-28T11:51:16.876792+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:51:16.876792+0000","last_undegraded":"2023-11-28T11:51:16.876792+0000","last_fullsized":"2023-11-28T11:51:16.876792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092461+0000","last_change":"2023-11-28T11:51:16.484243+0000","last_active":"2023-11-28T11:51:19.092461+0000","last_peered":"2023-11-28T11:51:19.092461+0000","last_clean":"2023-11-28T11:51:19.092461+0000","last_became_active":"2023-11-28T11:51:16.482924+0000","last_became_peered":"2023-11-28T11:51:16.482924+0000","last_unstale":"2023-11-28T11:51:19.092461+0000","last_undegraded":"2023-11-28T11:51:19.092461+0000","last_fullsized":"2023-11-28T11:51:19.092461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093787+0000","last_change":"2023-11-28T11:51:16.878987+0000","last_active":"2023-11-28T11:51:19.093787+0000","last_peered":"2023-11-28T11:51:19.093787+0000","last_clean":"2023-11-28T11:51:19.093787+0000","last_became_active":"2023-11-28T11:51:16.878798+0000","last_became_peered":"2023-11-28T11:51:16.878798+0000","last_unstale":"2023-11-28T11:51:19.093787+0000","last_undegraded":"2023-11-28T11:51:19.093787+0000","last_fullsized":"2023-11-28T11:51:19.093787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873448+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:51:16.873448+0000","last_peered":"2023-11-28T11:51:16.873448+0000","last_clean":"2023-11-28T11:51:16.873448+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:51:16.873448+0000","last_undegraded":"2023-11-28T11:51:16.873448+0000","last_fullsized":"2023-11-28T11:51:16.873448+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880298+0000","last_change":"2023-11-28T11:51:16.476786+0000","last_active":"2023-11-28T11:51:17.880298+0000","last_peered":"2023-11-28T11:51:17.880298+0000","last_clean":"2023-11-28T11:51:17.880298+0000","last_became_active":"2023-11-28T11:51:16.476460+0000","last_became_peered":"2023-11-28T11:51:16.476460+0000","last_unstale":"2023-11-28T11:51:17.880298+0000","last_undegraded":"2023-11-28T11:51:17.880298+0000","last_fullsized":"2023-11-28T11:51:17.880298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.094045+0000","last_change":"2023-11-28T11:51:16.890298+0000","last_active":"2023-11-28T11:51:19.094045+0000","last_peered":"2023-11-28T11:51:19.094045+0000","last_clean":"2023-11-28T11:51:19.094045+0000","last_became_active":"2023-11-28T11:51:16.890029+0000","last_became_peered":"2023-11-28T11:51:16.890029+0000","last_unstale":"2023-11-28T11:51:19.094045+0000","last_undegraded":"2023-11-28T11:51:19.094045+0000","last_fullsized":"2023-11-28T11:51:19.094045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:42.963230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873404+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:51:16.873404+0000","last_peered":"2023-11-28T11:51:16.873404+0000","last_clean":"2023-11-28T11:51:16.873404+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:51:16.873404+0000","last_undegraded":"2023-11-28T11:51:16.873404+0000","last_fullsized":"2023-11-28T11:51:16.873404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881318+0000","last_change":"2023-11-28T11:51:16.488913+0000","last_active":"2023-11-28T11:51:17.881318+0000","last_peered":"2023-11-28T11:51:17.881318+0000","last_clean":"2023-11-28T11:51:17.881318+0000","last_became_active":"2023-11-28T11:51:16.488559+0000","last_became_peered":"2023-11-28T11:51:16.488559+0000","last_unstale":"2023-11-28T11:51:17.881318+0000","last_undegraded":"2023-11-28T11:51:17.881318+0000","last_fullsized":"2023-11-28T11:51:17.881318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093581+0000","last_change":"2023-11-28T11:51:16.882735+0000","last_active":"2023-11-28T11:51:19.093581+0000","last_peered":"2023-11-28T11:51:19.093581+0000","last_clean":"2023-11-28T11:51:19.093581+0000","last_became_active":"2023-11-28T11:51:16.882478+0000","last_became_peered":"2023-11-28T11:51:16.882478+0000","last_unstale":"2023-11-28T11:51:19.093581+0000","last_undegraded":"2023-11-28T11:51:19.093581+0000","last_fullsized":"2023-11-28T11:51:19.093581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882570+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:51:16.882570+0000","last_peered":"2023-11-28T11:51:16.882570+0000","last_clean":"2023-11-28T11:51:16.882570+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:51:16.882570+0000","last_undegraded":"2023-11-28T11:51:16.882570+0000","last_fullsized":"2023-11-28T11:51:16.882570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093667+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:19.093667+0000","last_peered":"2023-11-28T11:51:19.093667+0000","last_clean":"2023-11-28T11:51:19.093667+0000","last_became_active":"2023-11-28T11:51:16.482544+0000","last_became_peered":"2023-11-28T11:51:16.482544+0000","last_unstale":"2023-11-28T11:51:19.093667+0000","last_undegraded":"2023-11-28T11:51:19.093667+0000","last_fullsized":"2023-11-28T11:51:19.093667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:40.095304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093117+0000","last_change":"2023-11-28T11:51:16.888841+0000","last_active":"2023-11-28T11:51:19.093117+0000","last_peered":"2023-11-28T11:51:19.093117+0000","last_clean":"2023-11-28T11:51:19.093117+0000","last_became_active":"2023-11-28T11:51:16.886560+0000","last_became_peered":"2023-11-28T11:51:16.886560+0000","last_unstale":"2023-11-28T11:51:19.093117+0000","last_undegraded":"2023-11-28T11:51:19.093117+0000","last_fullsized":"2023-11-28T11:51:19.093117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:15.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882346+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:51:16.882346+0000","last_peered":"2023-11-28T11:51:16.882346+0000","last_clean":"2023-11-28T11:51:16.882346+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:51:16.882346+0000","last_undegraded":"2023-11-28T11:51:16.882346+0000","last_fullsized":"2023-11-28T11:51:16.882346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879570+0000","last_change":"2023-11-28T11:51:16.475455+0000","last_active":"2023-11-28T11:51:17.879570+0000","last_peered":"2023-11-28T11:51:17.879570+0000","last_clean":"2023-11-28T11:51:17.879570+0000","last_became_active":"2023-11-28T11:51:16.475159+0000","last_became_peered":"2023-11-28T11:51:16.475159+0000","last_unstale":"2023-11-28T11:51:17.879570+0000","last_undegraded":"2023-11-28T11:51:17.879570+0000","last_fullsized":"2023-11-28T11:51:17.879570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093769+0000","last_change":"2023-11-28T11:51:16.880638+0000","last_active":"2023-11-28T11:51:19.093769+0000","last_peered":"2023-11-28T11:51:19.093769+0000","last_clean":"2023-11-28T11:51:19.093769+0000","last_became_active":"2023-11-28T11:51:16.880329+0000","last_became_peered":"2023-11-28T11:51:16.880329+0000","last_unstale":"2023-11-28T11:51:19.093769+0000","last_undegraded":"2023-11-28T11:51:19.093769+0000","last_fullsized":"2023-11-28T11:51:19.093769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092710+0000","last_change":"2023-11-28T11:51:16.490914+0000","last_active":"2023-11-28T11:51:19.092710+0000","last_peered":"2023-11-28T11:51:19.092710+0000","last_clean":"2023-11-28T11:51:19.092710+0000","last_became_active":"2023-11-28T11:51:16.490638+0000","last_became_peered":"2023-11-28T11:51:16.490638+0000","last_unstale":"2023-11-28T11:51:19.092710+0000","last_undegraded":"2023-11-28T11:51:19.092710+0000","last_fullsized":"2023-11-28T11:51:19.092710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:42.375209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092793+0000","last_change":"2023-11-28T11:51:16.891450+0000","last_active":"2023-11-28T11:51:19.092793+0000","last_peered":"2023-11-28T11:51:19.092793+0000","last_clean":"2023-11-28T11:51:19.092793+0000","last_became_active":"2023-11-28T11:51:16.891183+0000","last_became_peered":"2023-11-28T11:51:16.891183+0000","last_unstale":"2023-11-28T11:51:19.092793+0000","last_undegraded":"2023-11-28T11:51:19.092793+0000","last_fullsized":"2023-11-28T11:51:19.092793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:44.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879331+0000","last_change":"2023-11-28T11:51:16.478457+0000","last_active":"2023-11-28T11:51:17.879331+0000","last_peered":"2023-11-28T11:51:17.879331+0000","last_clean":"2023-11-28T11:51:17.879331+0000","last_became_active":"2023-11-28T11:51:16.478170+0000","last_became_peered":"2023-11-28T11:51:16.478170+0000","last_unstale":"2023-11-28T11:51:17.879331+0000","last_undegraded":"2023-11-28T11:51:17.879331+0000","last_fullsized":"2023-11-28T11:51:17.879331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881448+0000","last_change":"2023-11-28T11:51:16.885759+0000","last_active":"2023-11-28T11:51:17.881448+0000","last_peered":"2023-11-28T11:51:17.881448+0000","last_clean":"2023-11-28T11:51:17.881448+0000","last_became_active":"2023-11-28T11:51:16.885461+0000","last_became_peered":"2023-11-28T11:51:16.885461+0000","last_unstale":"2023-11-28T11:51:17.881448+0000","last_undegraded":"2023-11-28T11:51:17.881448+0000","last_fullsized":"2023-11-28T11:51:17.881448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879652+0000","last_change":"2023-11-28T11:51:16.478962+0000","last_active":"2023-11-28T11:51:17.879652+0000","last_peered":"2023-11-28T11:51:17.879652+0000","last_clean":"2023-11-28T11:51:17.879652+0000","last_became_active":"2023-11-28T11:51:16.478687+0000","last_became_peered":"2023-11-28T11:51:16.478687+0000","last_unstale":"2023-11-28T11:51:17.879652+0000","last_undegraded":"2023-11-28T11:51:17.879652+0000","last_fullsized":"2023-11-28T11:51:17.879652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881454+0000","last_change":"2023-11-28T11:51:16.891408+0000","last_active":"2023-11-28T11:51:17.881454+0000","last_peered":"2023-11-28T11:51:17.881454+0000","last_clean":"2023-11-28T11:51:17.881454+0000","last_became_active":"2023-11-28T11:51:16.891119+0000","last_became_peered":"2023-11-28T11:51:16.891119+0000","last_unstale":"2023-11-28T11:51:17.881454+0000","last_undegraded":"2023-11-28T11:51:17.881454+0000","last_fullsized":"2023-11-28T11:51:17.881454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880602+0000","last_change":"2023-11-28T11:51:16.483925+0000","last_active":"2023-11-28T11:51:17.880602+0000","last_peered":"2023-11-28T11:51:17.880602+0000","last_clean":"2023-11-28T11:51:17.880602+0000","last_became_active":"2023-11-28T11:51:16.483640+0000","last_became_peered":"2023-11-28T11:51:16.483640+0000","last_unstale":"2023-11-28T11:51:17.880602+0000","last_undegraded":"2023-11-28T11:51:17.880602+0000","last_fullsized":"2023-11-28T11:51:17.880602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880383+0000","last_change":"2023-11-28T11:51:16.884604+0000","last_active":"2023-11-28T11:51:17.880383+0000","last_peered":"2023-11-28T11:51:17.880383+0000","last_clean":"2023-11-28T11:51:17.880383+0000","last_became_active":"2023-11-28T11:51:16.884176+0000","last_became_peered":"2023-11-28T11:51:16.884176+0000","last_unstale":"2023-11-28T11:51:17.880383+0000","last_undegraded":"2023-11-28T11:51:17.880383+0000","last_fullsized":"2023-11-28T11:51:17.880383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093566+0000","last_change":"2023-11-28T11:51:16.488286+0000","last_active":"2023-11-28T11:51:19.093566+0000","last_peered":"2023-11-28T11:51:19.093566+0000","last_clean":"2023-11-28T11:51:19.093566+0000","last_became_active":"2023-11-28T11:51:16.487987+0000","last_became_peered":"2023-11-28T11:51:16.487987+0000","last_unstale":"2023-11-28T11:51:19.093566+0000","last_undegraded":"2023-11-28T11:51:19.093566+0000","last_fullsized":"2023-11-28T11:51:19.093566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:39:02.250990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880976+0000","last_change":"2023-11-28T11:51:16.888775+0000","last_active":"2023-11-28T11:51:17.880976+0000","last_peered":"2023-11-28T11:51:17.880976+0000","last_clean":"2023-11-28T11:51:17.880976+0000","last_became_active":"2023-11-28T11:51:16.888485+0000","last_became_peered":"2023-11-28T11:51:16.888485+0000","last_unstale":"2023-11-28T11:51:17.880976+0000","last_undegraded":"2023-11-28T11:51:17.880976+0000","last_fullsized":"2023-11-28T11:51:17.880976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:55.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093487+0000","last_change":"2023-11-28T11:51:16.888856+0000","last_active":"2023-11-28T11:51:19.093487+0000","last_peered":"2023-11-28T11:51:19.093487+0000","last_clean":"2023-11-28T11:51:19.093487+0000","last_became_active":"2023-11-28T11:51:16.887628+0000","last_became_peered":"2023-11-28T11:51:16.887628+0000","last_unstale":"2023-11-28T11:51:19.093487+0000","last_undegraded":"2023-11-28T11:51:19.093487+0000","last_fullsized":"2023-11-28T11:51:19.093487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880202+0000","last_change":"2023-11-28T11:51:16.887897+0000","last_active":"2023-11-28T11:51:17.880202+0000","last_peered":"2023-11-28T11:51:17.880202+0000","last_clean":"2023-11-28T11:51:17.880202+0000","last_became_active":"2023-11-28T11:51:16.887663+0000","last_became_peered":"2023-11-28T11:51:16.887663+0000","last_unstale":"2023-11-28T11:51:17.880202+0000","last_undegraded":"2023-11-28T11:51:17.880202+0000","last_fullsized":"2023-11-28T11:51:17.880202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881169+0000","last_change":"2023-11-28T11:51:16.891805+0000","last_active":"2023-11-28T11:51:17.881169+0000","last_peered":"2023-11-28T11:51:17.881169+0000","last_clean":"2023-11-28T11:51:17.881169+0000","last_became_active":"2023-11-28T11:51:16.891091+0000","last_became_peered":"2023-11-28T11:51:16.891091+0000","last_unstale":"2023-11-28T11:51:17.881169+0000","last_undegraded":"2023-11-28T11:51:17.881169+0000","last_fullsized":"2023-11-28T11:51:17.881169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880475+0000","last_change":"2023-11-28T11:51:16.897807+0000","last_active":"2023-11-28T11:51:17.880475+0000","last_peered":"2023-11-28T11:51:17.880475+0000","last_clean":"2023-11-28T11:51:17.880475+0000","last_became_active":"2023-11-28T11:51:16.897481+0000","last_became_peered":"2023-11-28T11:51:16.897481+0000","last_unstale":"2023-11-28T11:51:17.880475+0000","last_undegraded":"2023-11-28T11:51:17.880475+0000","last_fullsized":"2023-11-28T11:51:17.880475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093497+0000","last_change":"2023-11-28T11:51:16.886025+0000","last_active":"2023-11-28T11:51:19.093497+0000","last_peered":"2023-11-28T11:51:19.093497+0000","last_clean":"2023-11-28T11:51:19.093497+0000","last_became_active":"2023-11-28T11:51:16.885791+0000","last_became_peered":"2023-11-28T11:51:16.885791+0000","last_unstale":"2023-11-28T11:51:19.093497+0000","last_undegraded":"2023-11-28T11:51:19.093497+0000","last_fullsized":"2023-11-28T11:51:19.093497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9092,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362748,"statfs":{"total":96636764160,"available":96627453952,"internally_reserved":0,"allocated":847872,"data_stored":674887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9064,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362776,"statfs":{"total":96636764160,"available":96627482624,"internally_reserved":0,"allocated":819200,"data_stored":673123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8040,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363800,"statfs":{"total":96636764160,"available":96628531200,"internally_reserved":0,"allocated":229376,"data_stored":82845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":84359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7976,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363864,"statfs":{"total":96636764160,"available":96628596736,"internally_reserved":0,"allocated":229376,"data_stored":82405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:51:20.946 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:51:21.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.098+0000 7f6c85d47700 1 Processor -- start 2023-11-28T11:51:21.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.098+0000 7f6c85d47700 1 -- start start 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c85d47700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c8006b060 0x7f6c8006b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c85d47700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c800c9e10 con 0x7f6c80160b10 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c85d47700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8006efc0 con 0x7f6c80160740 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c85d47700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c800ca720 con 0x7f6c8006b060 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c7ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c8006b060 0x7f6c8006b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c7effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6c80160b10 0x7f6c800cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51160/0 (socket says 172.21.15.144:51160) 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c7effd700 1 -- 172.21.15.144:0/620638385 learned_addr learned my addr 172.21.15.144:0/620638385 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:21.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.099+0000 7f6c7ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c8006b060 0x7f6c8006b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34450/0 (socket says 172.21.15.144:34450) 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7ffff700 1 -- 172.21.15.144:0/620638385 >> v1:172.21.15.177:6789/0 conn(0x7f6c80160740 legacy=0x7f6c800cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7ffff700 1 -- 172.21.15.144:0/620638385 >> v1:172.21.15.144:6789/0 conn(0x7f6c80160b10 legacy=0x7f6c800cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7ffff700 1 -- 172.21.15.144:0/620638385 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c800ca0d0 con 0x7f6c8006b060 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7ffff700 1 --2- 172.21.15.144:0/620638385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8006b060 0x7f6c8006b4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009d90 tx=0x7f6c7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d67c340b8e92404 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7e7fc700 1 -- 172.21.15.144:0/620638385 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7800e070 con 0x7f6c8006b060 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7e7fc700 1 -- 172.21.15.144:0/620638385 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7800bbd0 con 0x7f6c8006b060 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c7e7fc700 1 -- 172.21.15.144:0/620638385 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78012430 con 0x7f6c8006b060 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 -- 172.21.15.144:0/620638385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8006b060 msgr2=0x7f6c8006b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 --2- 172.21.15.144:0/620638385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8006b060 0x7f6c8006b4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009d90 tx=0x7f6c7800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:21.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 -- 172.21.15.144:0/620638385 shutdown_connections 2023-11-28T11:51:21.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 --2- 172.21.15.144:0/620638385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8006b060 0x7f6c8006b4c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 -- 172.21.15.144:0/620638385 >> 172.21.15.144:0/620638385 conn(0x7f6c800b12b0 msgr2=0x7f6c800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:21.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 -- 172.21.15.144:0/620638385 shutdown_connections 2023-11-28T11:51:21.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.100+0000 7f6c85d47700 1 -- 172.21.15.144:0/620638385 wait complete. 2023-11-28T11:51:21.107 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.102+0000 7f6c85d47700 1 Processor -- start 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.102+0000 7f6c85d47700 1 -- start start 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.102+0000 7f6c85d47700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c8006b060 0x7f6c80157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.102+0000 7f6c85d47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 0x7f6c8015f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.102+0000 7f6c85d47700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c80160b10 0x7f6c8015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c85d47700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80158390 con 0x7f6c80160740 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c85d47700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8006cea0 con 0x7f6c80160b10 2023-11-28T11:51:21.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c85d47700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80159cf0 con 0x7f6c8006b060 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 0x7f6c8015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 0x7f6c8015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34454/0 (socket says 172.21.15.144:34454) 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 -- 172.21.15.144:0/137615442 learned_addr learned my addr 172.21.15.144:0/137615442 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 -- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c8006b060 msgr2=0x7f6c80157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c8006b060 0x7f6c80157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7ffff700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c80160b10 0x7f6c8015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 -- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c80160b10 msgr2=0x7f6c8015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c80160b10 0x7f6c8015f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 -- 172.21.15.144:0/137615442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009a70 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c7effd700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 0x7f6c8015f360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c4b0 tx=0x7f6c7000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70005d70 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700070d0 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c80160f60 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70007440 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.103+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c800ca3b0 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.104+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8006ec00 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.105+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c70019030 con 0x7f6c80160740 2023-11-28T11:51:21.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.105+0000 7f6c77fff700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c600682f0 0x7f6c6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.105+0000 7f6c7f7fe700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c600682f0 0x7f6c6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.105+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c70084e70 con 0x7f6c80160740 2023-11-28T11:51:21.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.105+0000 7f6c7f7fe700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c600682f0 0x7f6c6006a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c68000f60 tx=0x7f6c68007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.109+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c7004b5e0 con 0x7f6c80160740 2023-11-28T11:51:21.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.287+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c8015fde0 con 0x7f6c600682f0 2023-11-28T11:51:21.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.295+0000 7f6c77fff700 1 -- 172.21.15.144:0/137615442 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286891 (secure 0 0 0) 0x7f6c8015fde0 con 0x7f6c600682f0 2023-11-28T11:51:21.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:21.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c600682f0 msgr2=0x7f6c6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c600682f0 0x7f6c6006a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c68000f60 tx=0x7f6c68007a40 comp rx=0 tx=0).stop 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 msgr2=0x7f6c8015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 0x7f6c8015f360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c4b0 tx=0x7f6c7000c870 comp rx=0 tx=0).stop 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 shutdown_connections 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c600682f0 0x7f6c6006a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c80160b10 0x7f6c8015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c8006b060 0x7f6c80157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 --2- 172.21.15.144:0/137615442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c80160740 0x7f6c8015f360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 >> 172.21.15.144:0/137615442 conn(0x7f6c800b12b0 msgr2=0x7f6c8006b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:21.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 shutdown_connections 2023-11-28T11:51:21.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.298+0000 7f6c85d47700 1 -- 172.21.15.144:0/137615442 wait complete. 2023-11-28T11:51:21.306 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:51:21.311 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T11:51:20.482227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66016,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":754908704,"statfs":{"total":773094113280,"available":773026512896,"internally_reserved":0,"allocated":3047424,"data_stored":1845464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021964"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092580+0000","last_change":"2023-11-28T11:51:16.886968+0000","last_active":"2023-11-28T11:51:19.092580+0000","last_peered":"2023-11-28T11:51:19.092580+0000","last_clean":"2023-11-28T11:51:19.092580+0000","last_became_active":"2023-11-28T11:51:16.885939+0000","last_became_peered":"2023-11-28T11:51:16.885939+0000","last_unstale":"2023-11-28T11:51:19.092580+0000","last_undegraded":"2023-11-28T11:51:19.092580+0000","last_fullsized":"2023-11-28T11:51:19.092580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:14.596675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879462+0000","last_change":"2023-11-28T11:51:16.887419+0000","last_active":"2023-11-28T11:51:17.879462+0000","last_peered":"2023-11-28T11:51:17.879462+0000","last_clean":"2023-11-28T11:51:17.879462+0000","last_became_active":"2023-11-28T11:51:16.887120+0000","last_became_peered":"2023-11-28T11:51:16.887120+0000","last_unstale":"2023-11-28T11:51:17.879462+0000","last_undegraded":"2023-11-28T11:51:17.879462+0000","last_fullsized":"2023-11-28T11:51:17.879462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881469+0000","last_change":"2023-11-28T11:51:16.899556+0000","last_active":"2023-11-28T11:51:17.881469+0000","last_peered":"2023-11-28T11:51:17.881469+0000","last_clean":"2023-11-28T11:51:17.881469+0000","last_became_active":"2023-11-28T11:51:16.899088+0000","last_became_peered":"2023-11-28T11:51:16.899088+0000","last_unstale":"2023-11-28T11:51:17.881469+0000","last_undegraded":"2023-11-28T11:51:17.881469+0000","last_fullsized":"2023-11-28T11:51:17.881469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:23.190053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093312+0000","last_change":"2023-11-28T11:51:16.898501+0000","last_active":"2023-11-28T11:51:19.093312+0000","last_peered":"2023-11-28T11:51:19.093312+0000","last_clean":"2023-11-28T11:51:19.093312+0000","last_became_active":"2023-11-28T11:51:16.898263+0000","last_became_peered":"2023-11-28T11:51:16.898263+0000","last_unstale":"2023-11-28T11:51:19.093312+0000","last_undegraded":"2023-11-28T11:51:19.093312+0000","last_fullsized":"2023-11-28T11:51:19.093312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093023+0000","last_change":"2023-11-28T11:51:16.885205+0000","last_active":"2023-11-28T11:51:19.093023+0000","last_peered":"2023-11-28T11:51:19.093023+0000","last_clean":"2023-11-28T11:51:19.093023+0000","last_became_active":"2023-11-28T11:51:16.884964+0000","last_became_peered":"2023-11-28T11:51:16.884964+0000","last_unstale":"2023-11-28T11:51:19.093023+0000","last_undegraded":"2023-11-28T11:51:19.093023+0000","last_fullsized":"2023-11-28T11:51:19.093023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:36.307669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881261+0000","last_change":"2023-11-28T11:51:16.899331+0000","last_active":"2023-11-28T11:51:17.881261+0000","last_peered":"2023-11-28T11:51:17.881261+0000","last_clean":"2023-11-28T11:51:17.881261+0000","last_became_active":"2023-11-28T11:51:16.899026+0000","last_became_peered":"2023-11-28T11:51:16.899026+0000","last_unstale":"2023-11-28T11:51:17.881261+0000","last_undegraded":"2023-11-28T11:51:17.881261+0000","last_fullsized":"2023-11-28T11:51:17.881261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:40.625446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093069+0000","last_change":"2023-11-28T11:51:16.895279+0000","last_active":"2023-11-28T11:51:19.093069+0000","last_peered":"2023-11-28T11:51:19.093069+0000","last_clean":"2023-11-28T11:51:19.093069+0000","last_became_active":"2023-11-28T11:51:16.894949+0000","last_became_peered":"2023-11-28T11:51:16.894949+0000","last_unstale":"2023-11-28T11:51:19.093069+0000","last_undegraded":"2023-11-28T11:51:19.093069+0000","last_fullsized":"2023-11-28T11:51:19.093069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.882353+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.882010+0000","last_became_peered":"2023-11-28T11:51:16.882010+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093331+0000","last_change":"2023-11-28T11:51:16.885987+0000","last_active":"2023-11-28T11:51:19.093331+0000","last_peered":"2023-11-28T11:51:19.093331+0000","last_clean":"2023-11-28T11:51:19.093331+0000","last_became_active":"2023-11-28T11:51:16.885744+0000","last_became_peered":"2023-11-28T11:51:16.885744+0000","last_unstale":"2023-11-28T11:51:19.093331+0000","last_undegraded":"2023-11-28T11:51:19.093331+0000","last_fullsized":"2023-11-28T11:51:19.093331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:20:59.112675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879642+0000","last_change":"2023-11-28T11:51:16.898873+0000","last_active":"2023-11-28T11:51:17.879642+0000","last_peered":"2023-11-28T11:51:17.879642+0000","last_clean":"2023-11-28T11:51:17.879642+0000","last_became_active":"2023-11-28T11:51:16.898567+0000","last_became_peered":"2023-11-28T11:51:16.898567+0000","last_unstale":"2023-11-28T11:51:17.879642+0000","last_undegraded":"2023-11-28T11:51:17.879642+0000","last_fullsized":"2023-11-28T11:51:17.879642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881611+0000","last_change":"2023-11-28T11:51:16.892348+0000","last_active":"2023-11-28T11:51:17.881611+0000","last_peered":"2023-11-28T11:51:17.881611+0000","last_clean":"2023-11-28T11:51:17.881611+0000","last_became_active":"2023-11-28T11:51:16.891451+0000","last_became_peered":"2023-11-28T11:51:16.891451+0000","last_unstale":"2023-11-28T11:51:17.881611+0000","last_undegraded":"2023-11-28T11:51:17.881611+0000","last_fullsized":"2023-11-28T11:51:17.881611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:37.035204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092706+0000","last_change":"2023-11-28T11:51:16.893555+0000","last_active":"2023-11-28T11:51:19.092706+0000","last_peered":"2023-11-28T11:51:19.092706+0000","last_clean":"2023-11-28T11:51:19.092706+0000","last_became_active":"2023-11-28T11:51:16.893296+0000","last_became_peered":"2023-11-28T11:51:16.893296+0000","last_unstale":"2023-11-28T11:51:19.092706+0000","last_undegraded":"2023-11-28T11:51:19.092706+0000","last_fullsized":"2023-11-28T11:51:19.092706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:13.298585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092713+0000","last_change":"2023-11-28T11:51:16.885357+0000","last_active":"2023-11-28T11:51:19.092713+0000","last_peered":"2023-11-28T11:51:19.092713+0000","last_clean":"2023-11-28T11:51:19.092713+0000","last_became_active":"2023-11-28T11:51:16.885098+0000","last_became_peered":"2023-11-28T11:51:16.885098+0000","last_unstale":"2023-11-28T11:51:19.092713+0000","last_undegraded":"2023-11-28T11:51:19.092713+0000","last_fullsized":"2023-11-28T11:51:19.092713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:18.850790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880260+0000","last_change":"2023-11-28T11:51:16.882443+0000","last_active":"2023-11-28T11:51:17.880260+0000","last_peered":"2023-11-28T11:51:17.880260+0000","last_clean":"2023-11-28T11:51:17.880260+0000","last_became_active":"2023-11-28T11:51:16.882164+0000","last_became_peered":"2023-11-28T11:51:16.882164+0000","last_unstale":"2023-11-28T11:51:17.880260+0000","last_undegraded":"2023-11-28T11:51:17.880260+0000","last_fullsized":"2023-11-28T11:51:17.880260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093567+0000","last_change":"2023-11-28T11:51:16.895797+0000","last_active":"2023-11-28T11:51:19.093567+0000","last_peered":"2023-11-28T11:51:19.093567+0000","last_clean":"2023-11-28T11:51:19.093567+0000","last_became_active":"2023-11-28T11:51:16.895517+0000","last_became_peered":"2023-11-28T11:51:16.895517+0000","last_unstale":"2023-11-28T11:51:19.093567+0000","last_undegraded":"2023-11-28T11:51:19.093567+0000","last_fullsized":"2023-11-28T11:51:19.093567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881087+0000","last_change":"2023-11-28T11:51:16.899351+0000","last_active":"2023-11-28T11:51:17.881087+0000","last_peered":"2023-11-28T11:51:17.881087+0000","last_clean":"2023-11-28T11:51:17.881087+0000","last_became_active":"2023-11-28T11:51:16.899044+0000","last_became_peered":"2023-11-28T11:51:16.899044+0000","last_unstale":"2023-11-28T11:51:17.881087+0000","last_undegraded":"2023-11-28T11:51:17.881087+0000","last_fullsized":"2023-11-28T11:51:17.881087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:09.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093414+0000","last_change":"2023-11-28T11:51:16.885961+0000","last_active":"2023-11-28T11:51:19.093414+0000","last_peered":"2023-11-28T11:51:19.093414+0000","last_clean":"2023-11-28T11:51:19.093414+0000","last_became_active":"2023-11-28T11:51:16.885709+0000","last_became_peered":"2023-11-28T11:51:16.885709+0000","last_unstale":"2023-11-28T11:51:19.093414+0000","last_undegraded":"2023-11-28T11:51:19.093414+0000","last_fullsized":"2023-11-28T11:51:19.093414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:13.291091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881602+0000","last_change":"2023-11-28T11:51:16.893897+0000","last_active":"2023-11-28T11:51:17.881602+0000","last_peered":"2023-11-28T11:51:17.881602+0000","last_clean":"2023-11-28T11:51:17.881602+0000","last_became_active":"2023-11-28T11:51:16.893541+0000","last_became_peered":"2023-11-28T11:51:16.893541+0000","last_unstale":"2023-11-28T11:51:17.881602+0000","last_undegraded":"2023-11-28T11:51:17.881602+0000","last_fullsized":"2023-11-28T11:51:17.881602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092888+0000","last_change":"2023-11-28T11:51:16.887327+0000","last_active":"2023-11-28T11:51:19.092888+0000","last_peered":"2023-11-28T11:51:19.092888+0000","last_clean":"2023-11-28T11:51:19.092888+0000","last_became_active":"2023-11-28T11:51:16.887062+0000","last_became_peered":"2023-11-28T11:51:16.887062+0000","last_unstale":"2023-11-28T11:51:19.092888+0000","last_undegraded":"2023-11-28T11:51:19.092888+0000","last_fullsized":"2023-11-28T11:51:19.092888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:49.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881480+0000","last_change":"2023-11-28T11:51:16.889122+0000","last_active":"2023-11-28T11:51:17.881480+0000","last_peered":"2023-11-28T11:51:17.881480+0000","last_clean":"2023-11-28T11:51:17.881480+0000","last_became_active":"2023-11-28T11:51:16.888849+0000","last_became_peered":"2023-11-28T11:51:16.888849+0000","last_unstale":"2023-11-28T11:51:17.881480+0000","last_undegraded":"2023-11-28T11:51:17.881480+0000","last_fullsized":"2023-11-28T11:51:17.881480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093486+0000","last_change":"2023-11-28T11:51:16.895113+0000","last_active":"2023-11-28T11:51:19.093486+0000","last_peered":"2023-11-28T11:51:19.093486+0000","last_clean":"2023-11-28T11:51:19.093486+0000","last_became_active":"2023-11-28T11:51:16.894793+0000","last_became_peered":"2023-11-28T11:51:16.894793+0000","last_unstale":"2023-11-28T11:51:19.093486+0000","last_undegraded":"2023-11-28T11:51:19.093486+0000","last_fullsized":"2023-11-28T11:51:19.093486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898452+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897946+0000","last_became_peered":"2023-11-28T11:51:16.897946+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:20.222278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880183+0000","last_change":"2023-11-28T11:51:16.898903+0000","last_active":"2023-11-28T11:51:17.880183+0000","last_peered":"2023-11-28T11:51:17.880183+0000","last_clean":"2023-11-28T11:51:17.880183+0000","last_became_active":"2023-11-28T11:51:16.898719+0000","last_became_peered":"2023-11-28T11:51:16.898719+0000","last_unstale":"2023-11-28T11:51:17.880183+0000","last_undegraded":"2023-11-28T11:51:17.880183+0000","last_fullsized":"2023-11-28T11:51:17.880183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093156+0000","last_change":"2023-11-28T11:51:16.895329+0000","last_active":"2023-11-28T11:51:19.093156+0000","last_peered":"2023-11-28T11:51:19.093156+0000","last_clean":"2023-11-28T11:51:19.093156+0000","last_became_active":"2023-11-28T11:51:16.894998+0000","last_became_peered":"2023-11-28T11:51:16.894998+0000","last_unstale":"2023-11-28T11:51:19.093156+0000","last_undegraded":"2023-11-28T11:51:19.093156+0000","last_fullsized":"2023-11-28T11:51:19.093156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093690+0000","last_change":"2023-11-28T11:51:16.889063+0000","last_active":"2023-11-28T11:51:19.093690+0000","last_peered":"2023-11-28T11:51:19.093690+0000","last_clean":"2023-11-28T11:51:19.093690+0000","last_became_active":"2023-11-28T11:51:16.888711+0000","last_became_peered":"2023-11-28T11:51:16.888711+0000","last_unstale":"2023-11-28T11:51:19.093690+0000","last_undegraded":"2023-11-28T11:51:19.093690+0000","last_fullsized":"2023-11-28T11:51:19.093690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093545+0000","last_change":"2023-11-28T11:51:16.882290+0000","last_active":"2023-11-28T11:51:19.093545+0000","last_peered":"2023-11-28T11:51:19.093545+0000","last_clean":"2023-11-28T11:51:19.093545+0000","last_became_active":"2023-11-28T11:51:16.882049+0000","last_became_peered":"2023-11-28T11:51:16.882049+0000","last_unstale":"2023-11-28T11:51:19.093545+0000","last_undegraded":"2023-11-28T11:51:19.093545+0000","last_fullsized":"2023-11-28T11:51:19.093545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879435+0000","last_change":"2023-11-28T11:51:16.896720+0000","last_active":"2023-11-28T11:51:17.879435+0000","last_peered":"2023-11-28T11:51:17.879435+0000","last_clean":"2023-11-28T11:51:17.879435+0000","last_became_active":"2023-11-28T11:51:16.896429+0000","last_became_peered":"2023-11-28T11:51:16.896429+0000","last_unstale":"2023-11-28T11:51:17.879435+0000","last_undegraded":"2023-11-28T11:51:17.879435+0000","last_fullsized":"2023-11-28T11:51:17.879435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881315+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:17.881315+0000","last_peered":"2023-11-28T11:51:17.881315+0000","last_clean":"2023-11-28T11:51:17.881315+0000","last_became_active":"2023-11-28T11:51:16.482973+0000","last_became_peered":"2023-11-28T11:51:16.482973+0000","last_unstale":"2023-11-28T11:51:17.881315+0000","last_undegraded":"2023-11-28T11:51:17.881315+0000","last_fullsized":"2023-11-28T11:51:17.881315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880643+0000","last_change":"2023-11-28T11:51:16.887525+0000","last_active":"2023-11-28T11:51:17.880643+0000","last_peered":"2023-11-28T11:51:17.880643+0000","last_clean":"2023-11-28T11:51:17.880643+0000","last_became_active":"2023-11-28T11:51:16.887247+0000","last_became_peered":"2023-11-28T11:51:16.887247+0000","last_unstale":"2023-11-28T11:51:17.880643+0000","last_undegraded":"2023-11-28T11:51:17.880643+0000","last_fullsized":"2023-11-28T11:51:17.880643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093032+0000","last_change":"2023-11-28T11:51:16.481846+0000","last_active":"2023-11-28T11:51:19.093032+0000","last_peered":"2023-11-28T11:51:19.093032+0000","last_clean":"2023-11-28T11:51:19.093032+0000","last_became_active":"2023-11-28T11:51:16.481653+0000","last_became_peered":"2023-11-28T11:51:16.481653+0000","last_unstale":"2023-11-28T11:51:19.093032+0000","last_undegraded":"2023-11-28T11:51:19.093032+0000","last_fullsized":"2023-11-28T11:51:19.093032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:21.555920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093520+0000","last_change":"2023-11-28T11:51:16.895323+0000","last_active":"2023-11-28T11:51:19.093520+0000","last_peered":"2023-11-28T11:51:19.093520+0000","last_clean":"2023-11-28T11:51:19.093520+0000","last_became_active":"2023-11-28T11:51:16.895030+0000","last_became_peered":"2023-11-28T11:51:16.895030+0000","last_unstale":"2023-11-28T11:51:19.093520+0000","last_undegraded":"2023-11-28T11:51:19.093520+0000","last_fullsized":"2023-11-28T11:51:19.093520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:59.340661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880302+0000","last_change":"2023-11-28T11:51:16.488974+0000","last_active":"2023-11-28T11:51:17.880302+0000","last_peered":"2023-11-28T11:51:17.880302+0000","last_clean":"2023-11-28T11:51:17.880302+0000","last_became_active":"2023-11-28T11:51:16.488697+0000","last_became_peered":"2023-11-28T11:51:16.488697+0000","last_unstale":"2023-11-28T11:51:17.880302+0000","last_undegraded":"2023-11-28T11:51:17.880302+0000","last_fullsized":"2023-11-28T11:51:17.880302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093841+0000","last_change":"2023-11-28T11:51:16.888129+0000","last_active":"2023-11-28T11:51:19.093841+0000","last_peered":"2023-11-28T11:51:19.093841+0000","last_clean":"2023-11-28T11:51:19.093841+0000","last_became_active":"2023-11-28T11:51:16.887848+0000","last_became_peered":"2023-11-28T11:51:16.887848+0000","last_unstale":"2023-11-28T11:51:19.093841+0000","last_undegraded":"2023-11-28T11:51:19.093841+0000","last_fullsized":"2023-11-28T11:51:19.093841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:42.239688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093488+0000","last_change":"2023-11-28T11:51:16.488283+0000","last_active":"2023-11-28T11:51:19.093488+0000","last_peered":"2023-11-28T11:51:19.093488+0000","last_clean":"2023-11-28T11:51:19.093488+0000","last_became_active":"2023-11-28T11:51:16.487994+0000","last_became_peered":"2023-11-28T11:51:16.487994+0000","last_unstale":"2023-11-28T11:51:19.093488+0000","last_undegraded":"2023-11-28T11:51:19.093488+0000","last_fullsized":"2023-11-28T11:51:19.093488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:04.545703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880313+0000","last_change":"2023-11-28T11:51:16.891442+0000","last_active":"2023-11-28T11:51:17.880313+0000","last_peered":"2023-11-28T11:51:17.880313+0000","last_clean":"2023-11-28T11:51:17.880313+0000","last_became_active":"2023-11-28T11:51:16.891180+0000","last_became_peered":"2023-11-28T11:51:16.891180+0000","last_unstale":"2023-11-28T11:51:17.880313+0000","last_undegraded":"2023-11-28T11:51:17.880313+0000","last_fullsized":"2023-11-28T11:51:17.880313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879563+0000","last_change":"2023-11-28T11:51:16.484401+0000","last_active":"2023-11-28T11:51:17.879563+0000","last_peered":"2023-11-28T11:51:17.879563+0000","last_clean":"2023-11-28T11:51:17.879563+0000","last_became_active":"2023-11-28T11:51:16.484103+0000","last_became_peered":"2023-11-28T11:51:16.484103+0000","last_unstale":"2023-11-28T11:51:17.879563+0000","last_undegraded":"2023-11-28T11:51:17.879563+0000","last_fullsized":"2023-11-28T11:51:17.879563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880312+0000","last_change":"2023-11-28T11:51:16.898526+0000","last_active":"2023-11-28T11:51:17.880312+0000","last_peered":"2023-11-28T11:51:17.880312+0000","last_clean":"2023-11-28T11:51:17.880312+0000","last_became_active":"2023-11-28T11:51:16.897658+0000","last_became_peered":"2023-11-28T11:51:16.897658+0000","last_unstale":"2023-11-28T11:51:17.880312+0000","last_undegraded":"2023-11-28T11:51:17.880312+0000","last_fullsized":"2023-11-28T11:51:17.880312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093666+0000","last_change":"2023-11-28T11:51:16.484674+0000","last_active":"2023-11-28T11:51:19.093666+0000","last_peered":"2023-11-28T11:51:19.093666+0000","last_clean":"2023-11-28T11:51:19.093666+0000","last_became_active":"2023-11-28T11:51:16.484350+0000","last_became_peered":"2023-11-28T11:51:16.484350+0000","last_unstale":"2023-11-28T11:51:19.093666+0000","last_undegraded":"2023-11-28T11:51:19.093666+0000","last_fullsized":"2023-11-28T11:51:19.093666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:13.351037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093718+0000","last_change":"2023-11-28T11:51:16.889523+0000","last_active":"2023-11-28T11:51:19.093718+0000","last_peered":"2023-11-28T11:51:19.093718+0000","last_clean":"2023-11-28T11:51:19.093718+0000","last_became_active":"2023-11-28T11:51:16.889323+0000","last_became_peered":"2023-11-28T11:51:16.889323+0000","last_unstale":"2023-11-28T11:51:19.093718+0000","last_undegraded":"2023-11-28T11:51:19.093718+0000","last_fullsized":"2023-11-28T11:51:19.093718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881658+0000","last_change":"2023-11-28T11:51:16.482345+0000","last_active":"2023-11-28T11:51:17.881658+0000","last_peered":"2023-11-28T11:51:17.881658+0000","last_clean":"2023-11-28T11:51:17.881658+0000","last_became_active":"2023-11-28T11:51:16.482106+0000","last_became_peered":"2023-11-28T11:51:16.482106+0000","last_unstale":"2023-11-28T11:51:17.881658+0000","last_undegraded":"2023-11-28T11:51:17.881658+0000","last_fullsized":"2023-11-28T11:51:17.881658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881274+0000","last_change":"2023-11-28T11:51:16.890129+0000","last_active":"2023-11-28T11:51:17.881274+0000","last_peered":"2023-11-28T11:51:17.881274+0000","last_clean":"2023-11-28T11:51:17.881274+0000","last_became_active":"2023-11-28T11:51:16.889849+0000","last_became_peered":"2023-11-28T11:51:16.889849+0000","last_unstale":"2023-11-28T11:51:17.881274+0000","last_undegraded":"2023-11-28T11:51:17.881274+0000","last_fullsized":"2023-11-28T11:51:17.881274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093375+0000","last_change":"2023-11-28T11:51:16.485194+0000","last_active":"2023-11-28T11:51:19.093375+0000","last_peered":"2023-11-28T11:51:19.093375+0000","last_clean":"2023-11-28T11:51:19.093375+0000","last_became_active":"2023-11-28T11:51:16.484873+0000","last_became_peered":"2023-11-28T11:51:16.484873+0000","last_unstale":"2023-11-28T11:51:19.093375+0000","last_undegraded":"2023-11-28T11:51:19.093375+0000","last_fullsized":"2023-11-28T11:51:19.093375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880521+0000","last_change":"2023-11-28T11:51:16.887321+0000","last_active":"2023-11-28T11:51:17.880521+0000","last_peered":"2023-11-28T11:51:17.880521+0000","last_clean":"2023-11-28T11:51:17.880521+0000","last_became_active":"2023-11-28T11:51:16.886989+0000","last_became_peered":"2023-11-28T11:51:16.886989+0000","last_unstale":"2023-11-28T11:51:17.880521+0000","last_undegraded":"2023-11-28T11:51:17.880521+0000","last_fullsized":"2023-11-28T11:51:17.880521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093431+0000","last_change":"2023-11-28T11:51:16.486311+0000","last_active":"2023-11-28T11:51:19.093431+0000","last_peered":"2023-11-28T11:51:19.093431+0000","last_clean":"2023-11-28T11:51:19.093431+0000","last_became_active":"2023-11-28T11:51:16.485952+0000","last_became_peered":"2023-11-28T11:51:16.485952+0000","last_unstale":"2023-11-28T11:51:19.093431+0000","last_undegraded":"2023-11-28T11:51:19.093431+0000","last_fullsized":"2023-11-28T11:51:19.093431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:13.615057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879568+0000","last_change":"2023-11-28T11:51:16.896713+0000","last_active":"2023-11-28T11:51:17.879568+0000","last_peered":"2023-11-28T11:51:17.879568+0000","last_clean":"2023-11-28T11:51:17.879568+0000","last_became_active":"2023-11-28T11:51:16.896490+0000","last_became_peered":"2023-11-28T11:51:16.896490+0000","last_unstale":"2023-11-28T11:51:17.879568+0000","last_undegraded":"2023-11-28T11:51:17.879568+0000","last_fullsized":"2023-11-28T11:51:17.879568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093447+0000","last_change":"2023-11-28T11:51:16.485718+0000","last_active":"2023-11-28T11:51:19.093447+0000","last_peered":"2023-11-28T11:51:19.093447+0000","last_clean":"2023-11-28T11:51:19.093447+0000","last_became_active":"2023-11-28T11:51:16.485279+0000","last_became_peered":"2023-11-28T11:51:16.485279+0000","last_unstale":"2023-11-28T11:51:19.093447+0000","last_undegraded":"2023-11-28T11:51:19.093447+0000","last_fullsized":"2023-11-28T11:51:19.093447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898233+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897971+0000","last_became_peered":"2023-11-28T11:51:16.897971+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:59.713494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092990+0000","last_change":"2023-11-28T11:51:16.489032+0000","last_active":"2023-11-28T11:51:19.092990+0000","last_peered":"2023-11-28T11:51:19.092990+0000","last_clean":"2023-11-28T11:51:19.092990+0000","last_became_active":"2023-11-28T11:51:16.488752+0000","last_became_peered":"2023-11-28T11:51:16.488752+0000","last_unstale":"2023-11-28T11:51:19.092990+0000","last_undegraded":"2023-11-28T11:51:19.092990+0000","last_fullsized":"2023-11-28T11:51:19.092990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880771+0000","last_change":"2023-11-28T11:51:16.897679+0000","last_active":"2023-11-28T11:51:17.880771+0000","last_peered":"2023-11-28T11:51:17.880771+0000","last_clean":"2023-11-28T11:51:17.880771+0000","last_became_active":"2023-11-28T11:51:16.897336+0000","last_became_peered":"2023-11-28T11:51:16.897336+0000","last_unstale":"2023-11-28T11:51:17.880771+0000","last_undegraded":"2023-11-28T11:51:17.880771+0000","last_fullsized":"2023-11-28T11:51:17.880771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093751+0000","last_change":"2023-11-28T11:51:16.485545+0000","last_active":"2023-11-28T11:51:19.093751+0000","last_peered":"2023-11-28T11:51:19.093751+0000","last_clean":"2023-11-28T11:51:19.093751+0000","last_became_active":"2023-11-28T11:51:16.485097+0000","last_became_peered":"2023-11-28T11:51:16.485097+0000","last_unstale":"2023-11-28T11:51:19.093751+0000","last_undegraded":"2023-11-28T11:51:19.093751+0000","last_fullsized":"2023-11-28T11:51:19.093751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:41.232205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.896139+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.895569+0000","last_became_peered":"2023-11-28T11:51:16.895569+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093685+0000","last_change":"2023-11-28T11:51:16.485744+0000","last_active":"2023-11-28T11:51:19.093685+0000","last_peered":"2023-11-28T11:51:19.093685+0000","last_clean":"2023-11-28T11:51:19.093685+0000","last_became_active":"2023-11-28T11:51:16.485468+0000","last_became_peered":"2023-11-28T11:51:16.485468+0000","last_unstale":"2023-11-28T11:51:19.093685+0000","last_undegraded":"2023-11-28T11:51:19.093685+0000","last_fullsized":"2023-11-28T11:51:19.093685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881144+0000","last_change":"2023-11-28T11:51:16.887450+0000","last_active":"2023-11-28T11:51:17.881144+0000","last_peered":"2023-11-28T11:51:17.881144+0000","last_clean":"2023-11-28T11:51:17.881144+0000","last_became_active":"2023-11-28T11:51:16.887190+0000","last_became_peered":"2023-11-28T11:51:16.887190+0000","last_unstale":"2023-11-28T11:51:17.881144+0000","last_undegraded":"2023-11-28T11:51:17.881144+0000","last_fullsized":"2023-11-28T11:51:17.881144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:00.113811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093607+0000","last_change":"2023-11-28T11:51:16.485512+0000","last_active":"2023-11-28T11:51:19.093607+0000","last_peered":"2023-11-28T11:51:19.093607+0000","last_clean":"2023-11-28T11:51:19.093607+0000","last_became_active":"2023-11-28T11:51:16.485186+0000","last_became_peered":"2023-11-28T11:51:16.485186+0000","last_unstale":"2023-11-28T11:51:19.093607+0000","last_undegraded":"2023-11-28T11:51:19.093607+0000","last_fullsized":"2023-11-28T11:51:19.093607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:41.955746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093236+0000","last_change":"2023-11-28T11:51:16.888771+0000","last_active":"2023-11-28T11:51:19.093236+0000","last_peered":"2023-11-28T11:51:19.093236+0000","last_clean":"2023-11-28T11:51:19.093236+0000","last_became_active":"2023-11-28T11:51:16.888179+0000","last_became_peered":"2023-11-28T11:51:16.888179+0000","last_unstale":"2023-11-28T11:51:19.093236+0000","last_undegraded":"2023-11-28T11:51:19.093236+0000","last_fullsized":"2023-11-28T11:51:19.093236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879495+0000","last_change":"2023-11-28T11:51:16.475494+0000","last_active":"2023-11-28T11:51:17.879495+0000","last_peered":"2023-11-28T11:51:17.879495+0000","last_clean":"2023-11-28T11:51:17.879495+0000","last_became_active":"2023-11-28T11:51:16.475189+0000","last_became_peered":"2023-11-28T11:51:16.475189+0000","last_unstale":"2023-11-28T11:51:17.879495+0000","last_undegraded":"2023-11-28T11:51:17.879495+0000","last_fullsized":"2023-11-28T11:51:17.879495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092876+0000","last_change":"2023-11-28T11:51:16.889569+0000","last_active":"2023-11-28T11:51:19.092876+0000","last_peered":"2023-11-28T11:51:19.092876+0000","last_clean":"2023-11-28T11:51:19.092876+0000","last_became_active":"2023-11-28T11:51:16.888349+0000","last_became_peered":"2023-11-28T11:51:16.888349+0000","last_unstale":"2023-11-28T11:51:19.092876+0000","last_undegraded":"2023-11-28T11:51:19.092876+0000","last_fullsized":"2023-11-28T11:51:19.092876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:37.975550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880421+0000","last_change":"2023-11-28T11:51:16.491156+0000","last_active":"2023-11-28T11:51:17.880421+0000","last_peered":"2023-11-28T11:51:17.880421+0000","last_clean":"2023-11-28T11:51:17.880421+0000","last_became_active":"2023-11-28T11:51:16.490844+0000","last_became_peered":"2023-11-28T11:51:16.490844+0000","last_unstale":"2023-11-28T11:51:17.880421+0000","last_undegraded":"2023-11-28T11:51:17.880421+0000","last_fullsized":"2023-11-28T11:51:17.880421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881598+0000","last_change":"2023-11-28T11:51:16.890271+0000","last_active":"2023-11-28T11:51:17.881598+0000","last_peered":"2023-11-28T11:51:17.881598+0000","last_clean":"2023-11-28T11:51:17.881598+0000","last_became_active":"2023-11-28T11:51:16.889985+0000","last_became_peered":"2023-11-28T11:51:16.889985+0000","last_unstale":"2023-11-28T11:51:17.881598+0000","last_undegraded":"2023-11-28T11:51:17.881598+0000","last_fullsized":"2023-11-28T11:51:17.881598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:03.275473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093142+0000","last_change":"2023-11-28T11:51:16.482742+0000","last_active":"2023-11-28T11:51:19.093142+0000","last_peered":"2023-11-28T11:51:19.093142+0000","last_clean":"2023-11-28T11:51:19.093142+0000","last_became_active":"2023-11-28T11:51:16.482411+0000","last_became_peered":"2023-11-28T11:51:16.482411+0000","last_unstale":"2023-11-28T11:51:19.093142+0000","last_undegraded":"2023-11-28T11:51:19.093142+0000","last_fullsized":"2023-11-28T11:51:19.093142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881716+0000","last_change":"2023-11-28T11:51:16.893619+0000","last_active":"2023-11-28T11:51:17.881716+0000","last_peered":"2023-11-28T11:51:17.881716+0000","last_clean":"2023-11-28T11:51:17.881716+0000","last_became_active":"2023-11-28T11:51:16.893353+0000","last_became_peered":"2023-11-28T11:51:16.893353+0000","last_unstale":"2023-11-28T11:51:17.881716+0000","last_undegraded":"2023-11-28T11:51:17.881716+0000","last_fullsized":"2023-11-28T11:51:17.881716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881245+0000","last_change":"2023-11-28T11:51:16.480268+0000","last_active":"2023-11-28T11:51:17.881245+0000","last_peered":"2023-11-28T11:51:17.881245+0000","last_clean":"2023-11-28T11:51:17.881245+0000","last_became_active":"2023-11-28T11:51:16.479377+0000","last_became_peered":"2023-11-28T11:51:16.479377+0000","last_unstale":"2023-11-28T11:51:17.881245+0000","last_undegraded":"2023-11-28T11:51:17.881245+0000","last_fullsized":"2023-11-28T11:51:17.881245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881631+0000","last_change":"2023-11-28T11:51:16.890147+0000","last_active":"2023-11-28T11:51:17.881631+0000","last_peered":"2023-11-28T11:51:17.881631+0000","last_clean":"2023-11-28T11:51:17.881631+0000","last_became_active":"2023-11-28T11:51:16.889833+0000","last_became_peered":"2023-11-28T11:51:16.889833+0000","last_unstale":"2023-11-28T11:51:17.881631+0000","last_undegraded":"2023-11-28T11:51:17.881631+0000","last_fullsized":"2023-11-28T11:51:17.881631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880209+0000","last_change":"2023-11-28T11:51:16.478507+0000","last_active":"2023-11-28T11:51:17.880209+0000","last_peered":"2023-11-28T11:51:17.880209+0000","last_clean":"2023-11-28T11:51:17.880209+0000","last_became_active":"2023-11-28T11:51:16.478034+0000","last_became_peered":"2023-11-28T11:51:16.478034+0000","last_unstale":"2023-11-28T11:51:17.880209+0000","last_undegraded":"2023-11-28T11:51:17.880209+0000","last_fullsized":"2023-11-28T11:51:17.880209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093877+0000","last_change":"2023-11-28T11:51:16.885456+0000","last_active":"2023-11-28T11:51:19.093877+0000","last_peered":"2023-11-28T11:51:19.093877+0000","last_clean":"2023-11-28T11:51:19.093877+0000","last_became_active":"2023-11-28T11:51:16.885208+0000","last_became_peered":"2023-11-28T11:51:16.885208+0000","last_unstale":"2023-11-28T11:51:19.093877+0000","last_undegraded":"2023-11-28T11:51:19.093877+0000","last_fullsized":"2023-11-28T11:51:19.093877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:24.014435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880712+0000","last_change":"2023-11-28T11:51:16.483690+0000","last_active":"2023-11-28T11:51:17.880712+0000","last_peered":"2023-11-28T11:51:17.880712+0000","last_clean":"2023-11-28T11:51:17.880712+0000","last_became_active":"2023-11-28T11:51:16.483351+0000","last_became_peered":"2023-11-28T11:51:16.483351+0000","last_unstale":"2023-11-28T11:51:17.880712+0000","last_undegraded":"2023-11-28T11:51:17.880712+0000","last_fullsized":"2023-11-28T11:51:17.880712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877898+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:51:16.877898+0000","last_peered":"2023-11-28T11:51:16.877898+0000","last_clean":"2023-11-28T11:51:16.877898+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:51:16.877898+0000","last_undegraded":"2023-11-28T11:51:16.877898+0000","last_fullsized":"2023-11-28T11:51:16.877898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880121+0000","last_change":"2023-11-28T11:51:16.897901+0000","last_active":"2023-11-28T11:51:17.880121+0000","last_peered":"2023-11-28T11:51:17.880121+0000","last_clean":"2023-11-28T11:51:17.880121+0000","last_became_active":"2023-11-28T11:51:16.897621+0000","last_became_peered":"2023-11-28T11:51:16.897621+0000","last_unstale":"2023-11-28T11:51:17.880121+0000","last_undegraded":"2023-11-28T11:51:17.880121+0000","last_fullsized":"2023-11-28T11:51:17.880121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876559+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:51:16.876559+0000","last_peered":"2023-11-28T11:51:16.876559+0000","last_clean":"2023-11-28T11:51:16.876559+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:51:16.876559+0000","last_undegraded":"2023-11-28T11:51:16.876559+0000","last_fullsized":"2023-11-28T11:51:16.876559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093592+0000","last_change":"2023-11-28T11:51:16.486571+0000","last_active":"2023-11-28T11:51:19.093592+0000","last_peered":"2023-11-28T11:51:19.093592+0000","last_clean":"2023-11-28T11:51:19.093592+0000","last_became_active":"2023-11-28T11:51:16.486338+0000","last_became_peered":"2023-11-28T11:51:16.486338+0000","last_unstale":"2023-11-28T11:51:19.093592+0000","last_undegraded":"2023-11-28T11:51:19.093592+0000","last_fullsized":"2023-11-28T11:51:19.093592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:54.333661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881136+0000","last_change":"2023-11-28T11:51:16.889991+0000","last_active":"2023-11-28T11:51:17.881136+0000","last_peered":"2023-11-28T11:51:17.881136+0000","last_clean":"2023-11-28T11:51:17.881136+0000","last_became_active":"2023-11-28T11:51:16.889694+0000","last_became_peered":"2023-11-28T11:51:16.889694+0000","last_unstale":"2023-11-28T11:51:17.881136+0000","last_undegraded":"2023-11-28T11:51:17.881136+0000","last_fullsized":"2023-11-28T11:51:17.881136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.870699+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:51:16.870699+0000","last_peered":"2023-11-28T11:51:16.870699+0000","last_clean":"2023-11-28T11:51:16.870699+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:51:16.870699+0000","last_undegraded":"2023-11-28T11:51:16.870699+0000","last_fullsized":"2023-11-28T11:51:16.870699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877332+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:51:16.877332+0000","last_peered":"2023-11-28T11:51:16.877332+0000","last_clean":"2023-11-28T11:51:16.877332+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:51:16.877332+0000","last_undegraded":"2023-11-28T11:51:16.877332+0000","last_fullsized":"2023-11-28T11:51:16.877332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093112+0000","last_change":"2023-11-28T11:51:16.486764+0000","last_active":"2023-11-28T11:51:19.093112+0000","last_peered":"2023-11-28T11:51:19.093112+0000","last_clean":"2023-11-28T11:51:19.093112+0000","last_became_active":"2023-11-28T11:51:16.486448+0000","last_became_peered":"2023-11-28T11:51:16.486448+0000","last_unstale":"2023-11-28T11:51:19.093112+0000","last_undegraded":"2023-11-28T11:51:19.093112+0000","last_fullsized":"2023-11-28T11:51:19.093112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:15.029206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093385+0000","last_change":"2023-11-28T11:51:16.888008+0000","last_active":"2023-11-28T11:51:19.093385+0000","last_peered":"2023-11-28T11:51:19.093385+0000","last_clean":"2023-11-28T11:51:19.093385+0000","last_became_active":"2023-11-28T11:51:16.887745+0000","last_became_peered":"2023-11-28T11:51:16.887745+0000","last_unstale":"2023-11-28T11:51:19.093385+0000","last_undegraded":"2023-11-28T11:51:19.093385+0000","last_fullsized":"2023-11-28T11:51:19.093385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:39.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876792+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:51:16.876792+0000","last_peered":"2023-11-28T11:51:16.876792+0000","last_clean":"2023-11-28T11:51:16.876792+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:51:16.876792+0000","last_undegraded":"2023-11-28T11:51:16.876792+0000","last_fullsized":"2023-11-28T11:51:16.876792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092461+0000","last_change":"2023-11-28T11:51:16.484243+0000","last_active":"2023-11-28T11:51:19.092461+0000","last_peered":"2023-11-28T11:51:19.092461+0000","last_clean":"2023-11-28T11:51:19.092461+0000","last_became_active":"2023-11-28T11:51:16.482924+0000","last_became_peered":"2023-11-28T11:51:16.482924+0000","last_unstale":"2023-11-28T11:51:19.092461+0000","last_undegraded":"2023-11-28T11:51:19.092461+0000","last_fullsized":"2023-11-28T11:51:19.092461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093787+0000","last_change":"2023-11-28T11:51:16.878987+0000","last_active":"2023-11-28T11:51:19.093787+0000","last_peered":"2023-11-28T11:51:19.093787+0000","last_clean":"2023-11-28T11:51:19.093787+0000","last_became_active":"2023-11-28T11:51:16.878798+0000","last_became_peered":"2023-11-28T11:51:16.878798+0000","last_unstale":"2023-11-28T11:51:19.093787+0000","last_undegraded":"2023-11-28T11:51:19.093787+0000","last_fullsized":"2023-11-28T11:51:19.093787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873448+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:51:16.873448+0000","last_peered":"2023-11-28T11:51:16.873448+0000","last_clean":"2023-11-28T11:51:16.873448+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:51:16.873448+0000","last_undegraded":"2023-11-28T11:51:16.873448+0000","last_fullsized":"2023-11-28T11:51:16.873448+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880298+0000","last_change":"2023-11-28T11:51:16.476786+0000","last_active":"2023-11-28T11:51:17.880298+0000","last_peered":"2023-11-28T11:51:17.880298+0000","last_clean":"2023-11-28T11:51:17.880298+0000","last_became_active":"2023-11-28T11:51:16.476460+0000","last_became_peered":"2023-11-28T11:51:16.476460+0000","last_unstale":"2023-11-28T11:51:17.880298+0000","last_undegraded":"2023-11-28T11:51:17.880298+0000","last_fullsized":"2023-11-28T11:51:17.880298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.094045+0000","last_change":"2023-11-28T11:51:16.890298+0000","last_active":"2023-11-28T11:51:19.094045+0000","last_peered":"2023-11-28T11:51:19.094045+0000","last_clean":"2023-11-28T11:51:19.094045+0000","last_became_active":"2023-11-28T11:51:16.890029+0000","last_became_peered":"2023-11-28T11:51:16.890029+0000","last_unstale":"2023-11-28T11:51:19.094045+0000","last_undegraded":"2023-11-28T11:51:19.094045+0000","last_fullsized":"2023-11-28T11:51:19.094045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:42.963230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873404+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:51:16.873404+0000","last_peered":"2023-11-28T11:51:16.873404+0000","last_clean":"2023-11-28T11:51:16.873404+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:51:16.873404+0000","last_undegraded":"2023-11-28T11:51:16.873404+0000","last_fullsized":"2023-11-28T11:51:16.873404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881318+0000","last_change":"2023-11-28T11:51:16.488913+0000","last_active":"2023-11-28T11:51:17.881318+0000","last_peered":"2023-11-28T11:51:17.881318+0000","last_clean":"2023-11-28T11:51:17.881318+0000","last_became_active":"2023-11-28T11:51:16.488559+0000","last_became_peered":"2023-11-28T11:51:16.488559+0000","last_unstale":"2023-11-28T11:51:17.881318+0000","last_undegraded":"2023-11-28T11:51:17.881318+0000","last_fullsized":"2023-11-28T11:51:17.881318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093581+0000","last_change":"2023-11-28T11:51:16.882735+0000","last_active":"2023-11-28T11:51:19.093581+0000","last_peered":"2023-11-28T11:51:19.093581+0000","last_clean":"2023-11-28T11:51:19.093581+0000","last_became_active":"2023-11-28T11:51:16.882478+0000","last_became_peered":"2023-11-28T11:51:16.882478+0000","last_unstale":"2023-11-28T11:51:19.093581+0000","last_undegraded":"2023-11-28T11:51:19.093581+0000","last_fullsized":"2023-11-28T11:51:19.093581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882570+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:51:16.882570+0000","last_peered":"2023-11-28T11:51:16.882570+0000","last_clean":"2023-11-28T11:51:16.882570+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:51:16.882570+0000","last_undegraded":"2023-11-28T11:51:16.882570+0000","last_fullsized":"2023-11-28T11:51:16.882570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093667+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:19.093667+0000","last_peered":"2023-11-28T11:51:19.093667+0000","last_clean":"2023-11-28T11:51:19.093667+0000","last_became_active":"2023-11-28T11:51:16.482544+0000","last_became_peered":"2023-11-28T11:51:16.482544+0000","last_unstale":"2023-11-28T11:51:19.093667+0000","last_undegraded":"2023-11-28T11:51:19.093667+0000","last_fullsized":"2023-11-28T11:51:19.093667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:40.095304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093117+0000","last_change":"2023-11-28T11:51:16.888841+0000","last_active":"2023-11-28T11:51:19.093117+0000","last_peered":"2023-11-28T11:51:19.093117+0000","last_clean":"2023-11-28T11:51:19.093117+0000","last_became_active":"2023-11-28T11:51:16.886560+0000","last_became_peered":"2023-11-28T11:51:16.886560+0000","last_unstale":"2023-11-28T11:51:19.093117+0000","last_undegraded":"2023-11-28T11:51:19.093117+0000","last_fullsized":"2023-11-28T11:51:19.093117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:15.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882346+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:51:16.882346+0000","last_peered":"2023-11-28T11:51:16.882346+0000","last_clean":"2023-11-28T11:51:16.882346+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:51:16.882346+0000","last_undegraded":"2023-11-28T11:51:16.882346+0000","last_fullsized":"2023-11-28T11:51:16.882346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879570+0000","last_change":"2023-11-28T11:51:16.475455+0000","last_active":"2023-11-28T11:51:17.879570+0000","last_peered":"2023-11-28T11:51:17.879570+0000","last_clean":"2023-11-28T11:51:17.879570+0000","last_became_active":"2023-11-28T11:51:16.475159+0000","last_became_peered":"2023-11-28T11:51:16.475159+0000","last_unstale":"2023-11-28T11:51:17.879570+0000","last_undegraded":"2023-11-28T11:51:17.879570+0000","last_fullsized":"2023-11-28T11:51:17.879570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093769+0000","last_change":"2023-11-28T11:51:16.880638+0000","last_active":"2023-11-28T11:51:19.093769+0000","last_peered":"2023-11-28T11:51:19.093769+0000","last_clean":"2023-11-28T11:51:19.093769+0000","last_became_active":"2023-11-28T11:51:16.880329+0000","last_became_peered":"2023-11-28T11:51:16.880329+0000","last_unstale":"2023-11-28T11:51:19.093769+0000","last_undegraded":"2023-11-28T11:51:19.093769+0000","last_fullsized":"2023-11-28T11:51:19.093769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092710+0000","last_change":"2023-11-28T11:51:16.490914+0000","last_active":"2023-11-28T11:51:19.092710+0000","last_peered":"2023-11-28T11:51:19.092710+0000","last_clean":"2023-11-28T11:51:19.092710+0000","last_became_active":"2023-11-28T11:51:16.490638+0000","last_became_peered":"2023-11-28T11:51:16.490638+0000","last_unstale":"2023-11-28T11:51:19.092710+0000","last_undegraded":"2023-11-28T11:51:19.092710+0000","last_fullsized":"2023-11-28T11:51:19.092710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:42.375209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092793+0000","last_change":"2023-11-28T11:51:16.891450+0000","last_active":"2023-11-28T11:51:19.092793+0000","last_peered":"2023-11-28T11:51:19.092793+0000","last_clean":"2023-11-28T11:51:19.092793+0000","last_became_active":"2023-11-28T11:51:16.891183+0000","last_became_peered":"2023-11-28T11:51:16.891183+0000","last_unstale":"2023-11-28T11:51:19.092793+0000","last_undegraded":"2023-11-28T11:51:19.092793+0000","last_fullsized":"2023-11-28T11:51:19.092793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:44.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879331+0000","last_change":"2023-11-28T11:51:16.478457+0000","last_active":"2023-11-28T11:51:17.879331+0000","last_peered":"2023-11-28T11:51:17.879331+0000","last_clean":"2023-11-28T11:51:17.879331+0000","last_became_active":"2023-11-28T11:51:16.478170+0000","last_became_peered":"2023-11-28T11:51:16.478170+0000","last_unstale":"2023-11-28T11:51:17.879331+0000","last_undegraded":"2023-11-28T11:51:17.879331+0000","last_fullsized":"2023-11-28T11:51:17.879331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881448+0000","last_change":"2023-11-28T11:51:16.885759+0000","last_active":"2023-11-28T11:51:17.881448+0000","last_peered":"2023-11-28T11:51:17.881448+0000","last_clean":"2023-11-28T11:51:17.881448+0000","last_became_active":"2023-11-28T11:51:16.885461+0000","last_became_peered":"2023-11-28T11:51:16.885461+0000","last_unstale":"2023-11-28T11:51:17.881448+0000","last_undegraded":"2023-11-28T11:51:17.881448+0000","last_fullsized":"2023-11-28T11:51:17.881448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879652+0000","last_change":"2023-11-28T11:51:16.478962+0000","last_active":"2023-11-28T11:51:17.879652+0000","last_peered":"2023-11-28T11:51:17.879652+0000","last_clean":"2023-11-28T11:51:17.879652+0000","last_became_active":"2023-11-28T11:51:16.478687+0000","last_became_peered":"2023-11-28T11:51:16.478687+0000","last_unstale":"2023-11-28T11:51:17.879652+0000","last_undegraded":"2023-11-28T11:51:17.879652+0000","last_fullsized":"2023-11-28T11:51:17.879652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881454+0000","last_change":"2023-11-28T11:51:16.891408+0000","last_active":"2023-11-28T11:51:17.881454+0000","last_peered":"2023-11-28T11:51:17.881454+0000","last_clean":"2023-11-28T11:51:17.881454+0000","last_became_active":"2023-11-28T11:51:16.891119+0000","last_became_peered":"2023-11-28T11:51:16.891119+0000","last_unstale":"2023-11-28T11:51:17.881454+0000","last_undegraded":"2023-11-28T11:51:17.881454+0000","last_fullsized":"2023-11-28T11:51:17.881454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880602+0000","last_change":"2023-11-28T11:51:16.483925+0000","last_active":"2023-11-28T11:51:17.880602+0000","last_peered":"2023-11-28T11:51:17.880602+0000","last_clean":"2023-11-28T11:51:17.880602+0000","last_became_active":"2023-11-28T11:51:16.483640+0000","last_became_peered":"2023-11-28T11:51:16.483640+0000","last_unstale":"2023-11-28T11:51:17.880602+0000","last_undegraded":"2023-11-28T11:51:17.880602+0000","last_fullsized":"2023-11-28T11:51:17.880602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880383+0000","last_change":"2023-11-28T11:51:16.884604+0000","last_active":"2023-11-28T11:51:17.880383+0000","last_peered":"2023-11-28T11:51:17.880383+0000","last_clean":"2023-11-28T11:51:17.880383+0000","last_became_active":"2023-11-28T11:51:16.884176+0000","last_became_peered":"2023-11-28T11:51:16.884176+0000","last_unstale":"2023-11-28T11:51:17.880383+0000","last_undegraded":"2023-11-28T11:51:17.880383+0000","last_fullsized":"2023-11-28T11:51:17.880383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093566+0000","last_change":"2023-11-28T11:51:16.488286+0000","last_active":"2023-11-28T11:51:19.093566+0000","last_peered":"2023-11-28T11:51:19.093566+0000","last_clean":"2023-11-28T11:51:19.093566+0000","last_became_active":"2023-11-28T11:51:16.487987+0000","last_became_peered":"2023-11-28T11:51:16.487987+0000","last_unstale":"2023-11-28T11:51:19.093566+0000","last_undegraded":"2023-11-28T11:51:19.093566+0000","last_fullsized":"2023-11-28T11:51:19.093566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:39:02.250990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880976+0000","last_change":"2023-11-28T11:51:16.888775+0000","last_active":"2023-11-28T11:51:17.880976+0000","last_peered":"2023-11-28T11:51:17.880976+0000","last_clean":"2023-11-28T11:51:17.880976+0000","last_became_active":"2023-11-28T11:51:16.888485+0000","last_became_peered":"2023-11-28T11:51:16.888485+0000","last_unstale":"2023-11-28T11:51:17.880976+0000","last_undegraded":"2023-11-28T11:51:17.880976+0000","last_fullsized":"2023-11-28T11:51:17.880976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:55.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093487+0000","last_change":"2023-11-28T11:51:16.888856+0000","last_active":"2023-11-28T11:51:19.093487+0000","last_peered":"2023-11-28T11:51:19.093487+0000","last_clean":"2023-11-28T11:51:19.093487+0000","last_became_active":"2023-11-28T11:51:16.887628+0000","last_became_peered":"2023-11-28T11:51:16.887628+0000","last_unstale":"2023-11-28T11:51:19.093487+0000","last_undegraded":"2023-11-28T11:51:19.093487+0000","last_fullsized":"2023-11-28T11:51:19.093487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880202+0000","last_change":"2023-11-28T11:51:16.887897+0000","last_active":"2023-11-28T11:51:17.880202+0000","last_peered":"2023-11-28T11:51:17.880202+0000","last_clean":"2023-11-28T11:51:17.880202+0000","last_became_active":"2023-11-28T11:51:16.887663+0000","last_became_peered":"2023-11-28T11:51:16.887663+0000","last_unstale":"2023-11-28T11:51:17.880202+0000","last_undegraded":"2023-11-28T11:51:17.880202+0000","last_fullsized":"2023-11-28T11:51:17.880202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881169+0000","last_change":"2023-11-28T11:51:16.891805+0000","last_active":"2023-11-28T11:51:17.881169+0000","last_peered":"2023-11-28T11:51:17.881169+0000","last_clean":"2023-11-28T11:51:17.881169+0000","last_became_active":"2023-11-28T11:51:16.891091+0000","last_became_peered":"2023-11-28T11:51:16.891091+0000","last_unstale":"2023-11-28T11:51:17.881169+0000","last_undegraded":"2023-11-28T11:51:17.881169+0000","last_fullsized":"2023-11-28T11:51:17.881169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880475+0000","last_change":"2023-11-28T11:51:16.897807+0000","last_active":"2023-11-28T11:51:17.880475+0000","last_peered":"2023-11-28T11:51:17.880475+0000","last_clean":"2023-11-28T11:51:17.880475+0000","last_became_active":"2023-11-28T11:51:16.897481+0000","last_became_peered":"2023-11-28T11:51:16.897481+0000","last_unstale":"2023-11-28T11:51:17.880475+0000","last_undegraded":"2023-11-28T11:51:17.880475+0000","last_fullsized":"2023-11-28T11:51:17.880475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093497+0000","last_change":"2023-11-28T11:51:16.886025+0000","last_active":"2023-11-28T11:51:19.093497+0000","last_peered":"2023-11-28T11:51:19.093497+0000","last_clean":"2023-11-28T11:51:19.093497+0000","last_became_active":"2023-11-28T11:51:16.885791+0000","last_became_peered":"2023-11-28T11:51:16.885791+0000","last_unstale":"2023-11-28T11:51:19.093497+0000","last_undegraded":"2023-11-28T11:51:19.093497+0000","last_fullsized":"2023-11-28T11:51:19.093497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9092,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362748,"statfs":{"total":96636764160,"available":96627453952,"internally_reserved":0,"allocated":847872,"data_stored":674887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9064,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362776,"statfs":{"total":96636764160,"available":96627482624,"internally_reserved":0,"allocated":819200,"data_stored":673123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7972,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363868,"statfs":{"total":96636764160,"available":96628600832,"internally_reserved":0,"allocated":225280,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8040,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363800,"statfs":{"total":96636764160,"available":96628531200,"internally_reserved":0,"allocated":229376,"data_stored":82845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":84359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7976,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363864,"statfs":{"total":96636764160,"available":96628596736,"internally_reserved":0,"allocated":229376,"data_stored":82405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:51:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:51:21.316 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T11:51:21.316 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T11:51:21.316 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T11:51:21.317 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T11:51:21.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.472+0000 7ffb72f00700 1 Processor -- start 2023-11-28T11:51:21.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.472+0000 7ffb72f00700 1 -- start start 2023-11-28T11:51:21.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb72f00700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffb6c071040 0x7ffb6c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb72f00700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb6c06dee0 0x7ffb6c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb72f00700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c06cdc0 0x7ffb6c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb72f00700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c071950 con 0x7ffb6c06dee0 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb72f00700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c15d060 con 0x7ffb6c071040 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb72f00700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c15d340 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb6bfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb6c06dee0 0x7ffb6c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb6bfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb6c06dee0 0x7ffb6c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34460/0 (socket says 172.21.15.144:34460) 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb6bfff700 1 -- 172.21.15.144:0/3207587134 learned_addr learned my addr 172.21.15.144:0/3207587134 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb70c9c700 1 --2- 172.21.15.144:0/3207587134 >> v2:172.21.15.177:3300/0 conn(0x7ffb6c071040 0x7ffb6c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb7149d700 1 --2- 172.21.15.144:0/3207587134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c06cdc0 0x7ffb6c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb6bfff700 1 -- 172.21.15.144:0/3207587134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c06cdc0 msgr2=0x7ffb6c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.473+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/3207587134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c06cdc0 0x7ffb6c06d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6bfff700 1 -- 172.21.15.144:0/3207587134 >> v2:172.21.15.177:3300/0 conn(0x7ffb6c071040 msgr2=0x7ffb6c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/3207587134 >> v2:172.21.15.177:3300/0 conn(0x7ffb6c071040 0x7ffb6c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6bfff700 1 -- 172.21.15.144:0/3207587134 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c15d620 con 0x7ffb6c06dee0 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/3207587134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c06e330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0d0f50 tx=0x7ffb58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc9c1192662896 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6b7fe700 1 -- 172.21.15.144:0/3207587134 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800b870 con 0x7ffb6c06dee0 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6b7fe700 1 -- 172.21.15.144:0/3207587134 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800b9d0 con 0x7ffb6c06dee0 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb6b7fe700 1 -- 172.21.15.144:0/3207587134 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800bd60 con 0x7ffb6c06dee0 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 -- 172.21.15.144:0/3207587134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06dee0 msgr2=0x7ffb6c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 --2- 172.21.15.144:0/3207587134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c06e330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0d0f50 tx=0x7ffb58009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 -- 172.21.15.144:0/3207587134 shutdown_connections 2023-11-28T11:51:21.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 --2- 172.21.15.144:0/3207587134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c06cdc0 0x7ffb6c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 --2- 172.21.15.144:0/3207587134 >> v2:172.21.15.177:3300/0 conn(0x7ffb6c071040 0x7ffb6c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 --2- 172.21.15.144:0/3207587134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c06e330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 -- 172.21.15.144:0/3207587134 >> 172.21.15.144:0/3207587134 conn(0x7ffb6c0d0000 msgr2=0x7ffb6c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:21.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 -- 172.21.15.144:0/3207587134 shutdown_connections 2023-11-28T11:51:21.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.474+0000 7ffb72f00700 1 -- 172.21.15.144:0/3207587134 wait complete. 2023-11-28T11:51:21.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 Processor -- start 2023-11-28T11:51:21.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 -- start start 2023-11-28T11:51:21.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 0x7ffb6c0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c071040 0x7ffb6c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0c6e90 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.476+0000 7ffb72f00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0c3e10 con 0x7ffb6c06dee0 2023-11-28T11:51:21.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb72f00700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c159360 con 0x7ffb6c071040 2023-11-28T11:51:21.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 0x7ffb6c0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 0x7ffb6c0b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34464/0 (socket says 172.21.15.144:34464) 2023-11-28T11:51:21.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 -- 172.21.15.144:0/2885149811 learned_addr learned my addr 172.21.15.144:0/2885149811 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:21.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb7149d700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c071040 0x7ffb6c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 -- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c071040 msgr2=0x7ffb6c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c071040 0x7ffb6c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 -- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb6c06dee0 msgr2=0x7ffb6c145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 -- 172.21.15.144:0/2885149811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009550 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:21.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb7149d700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c071040 0x7ffb6c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:21.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb70c9c700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 0x7ffb6c0b5820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000d800 tx=0x7ffb6000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600125f0 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60012750 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c06e870 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60012b00 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.477+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c15d900 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.478+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c15ccd0 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.479+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb60012c60 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.479+0000 7ffb697fa700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffb500682f0 0x7ffb5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.479+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffb500682f0 0x7ffb5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.479+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb600861b0 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.479+0000 7ffb6bfff700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffb500682f0 0x7ffb5006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009840 tx=0x7ffb58008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.483+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb60052a10 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.719+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ffb6c0720a0 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.722+0000 7ffb697fa700 1 -- 172.21.15.144:0/2885149811 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166384 (secure 0 0 0) 0x7ffb60056030 con 0x7ffb6c06cdc0 2023-11-28T11:51:21.725 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffb500682f0 msgr2=0x7ffb5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffb500682f0 0x7ffb5006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009840 tx=0x7ffb58008030 comp rx=0 tx=0).stop 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 msgr2=0x7ffb6c0b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 0x7ffb6c0b5820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000d800 tx=0x7ffb6000dbc0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 shutdown_connections 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c06cdc0 0x7ffb6c0b5820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffb500682f0 0x7ffb5006a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb6c071040 0x7ffb6c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 --2- 172.21.15.144:0/2885149811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb6c06dee0 0x7ffb6c145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 >> 172.21.15.144:0/2885149811 conn(0x7ffb6c0d0000 msgr2=0x7ffb6c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:21.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 shutdown_connections 2023-11-28T11:51:21.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.725+0000 7ffb72f00700 1 -- 172.21.15.144:0/2885149811 wait complete. 2023-11-28T11:51:21.738 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":103287},{"type":"v1","addr":"172.21.15.144:6833","nonce":103287}]},"active_addr":"172.21.15.144:6833/103287","active_change":"2023-11-28T11:51:04.474629+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":535826360}]},{"addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":649076024}]},{"addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":4082462551}]},{"addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2090165158}]}]} 2023-11-28T11:51:21.741 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T11:51:21.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T11:51:21.741 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:21.860 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.859+0000 7fbe0f945700 1 Processor -- start 2023-11-28T11:51:21.860 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.860+0000 7fbe0f945700 1 -- start start 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.860+0000 7fbe0f945700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbe0806dee0 0x7fbe0806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.860+0000 7fbe0f945700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0815d060 con 0x7fbe08160540 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.860+0000 7fbe0f945700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0815d340 con 0x7fbe08071040 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.860+0000 7fbe0f945700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0815d620 con 0x7fbe0806dee0 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0dee2700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbe08160540 0x7fbe08160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51186/0 (socket says 172.21.15.144:51186) 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0dee2700 1 -- 172.21.15.144:0/3977419203 learned_addr learned my addr 172.21.15.144:0/3977419203 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0cee0700 1 --2- 172.21.15.144:0/3977419203 >> v2:172.21.15.177:3300/0 conn(0x7fbe0806dee0 0x7fbe0806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbdfffff700 1 -- 172.21.15.144:0/3977419203 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244630403 0 0) 0x7fbe0815d060 con 0x7fbe08160540 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbdfffff700 1 -- 172.21.15.144:0/3977419203 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf40035b0 con 0x7fbe08160540 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0cee0700 1 -- 172.21.15.144:0/3977419203 >> v1:172.21.15.177:6789/0 conn(0x7fbe08071040 legacy=0x7fbe08071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbdfffff700 1 -- 172.21.15.144:0/3977419203 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029834721 0 0) 0x7fbdf40035b0 con 0x7fbe08160540 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0cee0700 1 -- 172.21.15.144:0/3977419203 >> v1:172.21.15.144:6789/0 conn(0x7fbe08160540 legacy=0x7fbe08160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0cee0700 1 -- 172.21.15.144:0/3977419203 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0815d900 con 0x7fbe0806dee0 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbe0cee0700 1 --2- 172.21.15.144:0/3977419203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe0806dee0 0x7fbe0806e330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009840 tx=0x7fbe04009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2395d9ddf7fadad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbdfffff700 1 -- 172.21.15.144:0/3977419203 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000019f0 con 0x7fbe0806dee0 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbdfffff700 1 -- 172.21.15.144:0/3977419203 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe04004030 con 0x7fbe0806dee0 2023-11-28T11:51:21.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.861+0000 7fbdfffff700 1 -- 172.21.15.144:0/3977419203 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe040043c0 con 0x7fbe0806dee0 2023-11-28T11:51:21.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 -- 172.21.15.144:0/3977419203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe0806dee0 msgr2=0x7fbe0806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 --2- 172.21.15.144:0/3977419203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe0806dee0 0x7fbe0806e330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009840 tx=0x7fbe04009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 -- 172.21.15.144:0/3977419203 shutdown_connections 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 --2- 172.21.15.144:0/3977419203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe0806dee0 0x7fbe0806e330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 -- 172.21.15.144:0/3977419203 >> 172.21.15.144:0/3977419203 conn(0x7fbe080d0000 msgr2=0x7fbe080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 -- 172.21.15.144:0/3977419203 shutdown_connections 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.862+0000 7fbe0f945700 1 -- 172.21.15.144:0/3977419203 wait complete. 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.863+0000 7fbe0f945700 1 Processor -- start 2023-11-28T11:51:21.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.863+0000 7fbe0f945700 1 -- start start 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0f945700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe0806dee0 0x7fbe0815fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0f945700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 0x7fbe081555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0f945700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe08160540 0x7fbe080b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0f945700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081600c0 con 0x7fbe08071040 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0f945700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08155b30 con 0x7fbe08160540 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0f945700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080c67c0 con 0x7fbe0806dee0 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 0x7fbe081555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 0x7fbe081555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34480/0 (socket says 172.21.15.144:34480) 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 -- 172.21.15.144:0/178351142 learned_addr learned my addr 172.21.15.144:0/178351142 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:21.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0d6e1700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe0806dee0 0x7fbe0815fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0dee2700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe08160540 0x7fbe080b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 -- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe0806dee0 msgr2=0x7fbe0815fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe0806dee0 0x7fbe0815fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 -- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe08160540 msgr2=0x7fbe080b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe08160540 0x7fbe080b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0cee0700 1 -- 172.21.15.144:0/178351142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe040094a0 con 0x7fbe08071040 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.864+0000 7fbe0dee2700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe08160540 0x7fbe080b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.865+0000 7fbe0cee0700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 0x7fbe081555f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe0806c7b0 tx=0x7fbe04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.865+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe04003d60 con 0x7fbe08071040 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.865+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe04010590 con 0x7fbe08071040 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.865+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0815ccd0 con 0x7fbe08071040 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.865+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe04010940 con 0x7fbe08071040 2023-11-28T11:51:21.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.865+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0806e870 con 0x7fbe08071040 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.866+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0806a270 con 0x7fbe08071040 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.866+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe0400f050 con 0x7fbe08071040 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.866+0000 7fbdfdffb700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbdec068320 0x7fbdec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.867+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe040872c0 con 0x7fbe08071040 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.867+0000 7fbe0d6e1700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbdec068320 0x7fbdec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.867+0000 7fbe0d6e1700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbdec068320 0x7fbdec06a7d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80012e0 tx=0x7fbdf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:21.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:21.871+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe04053b20 con 0x7fbe08071040 2023-11-28T11:51:22.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.057+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe080720a0 con 0x7fbe08071040 2023-11-28T11:51:22.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.058+0000 7fbdfdffb700 1 -- 172.21.15.144:0/178351142 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fbe04023360 con 0x7fbe08071040 2023-11-28T11:51:22.059 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:22.059 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:22.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbdec068320 msgr2=0x7fbdec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbdec068320 0x7fbdec06a7d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80012e0 tx=0x7fbdf8009040 comp rx=0 tx=0).stop 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 msgr2=0x7fbe081555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 0x7fbe081555f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe0806c7b0 tx=0x7fbe04004750 comp rx=0 tx=0).stop 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 shutdown_connections 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbdec068320 0x7fbdec06a7d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe08160540 0x7fbe080b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe0806dee0 0x7fbe0815fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 --2- 172.21.15.144:0/178351142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe08071040 0x7fbe081555f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 >> 172.21.15.144:0/178351142 conn(0x7fbe080d0000 msgr2=0x7fbe080cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 shutdown_connections 2023-11-28T11:51:22.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.060+0000 7fbe0f945700 1 -- 172.21.15.144:0/178351142 wait complete. 2023-11-28T11:51:22.073 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T11:51:22.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:22.194 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.193+0000 7f1f1d8cb700 1 Processor -- start 2023-11-28T11:51:22.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f1d8cb700 1 -- start start 2023-11-28T11:51:22.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f1d8cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f180b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f1d8cb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f180d1950 con 0x7f1f180cd7c0 2023-11-28T11:51:22.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f1d8cb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f180d24f0 con 0x7f1f18161340 2023-11-28T11:51:22.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f1d8cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180b5800 con 0x7f1f1815c9b0 2023-11-28T11:51:22.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f177fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1f180cd7c0 0x7f1f180b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51200/0 (socket says 172.21.15.144:51200) 2023-11-28T11:51:22.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.194+0000 7f1f177fe700 1 -- 172.21.15.144:0/1062043327 learned_addr learned my addr 172.21.15.144:0/1062043327 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f16ffd700 1 --2- 172.21.15.144:0/1062043327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f180b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454275035 0 0) 0x7f1f180d1950 con 0x7f1f180cd7c0 2023-11-28T11:51:22.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f000035b0 con 0x7f1f180cd7c0 2023-11-28T11:51:22.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238868512 0 0) 0x7f1f180d24f0 con 0x7f1f18161340 2023-11-28T11:51:22.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f180d1950 con 0x7f1f18161340 2023-11-28T11:51:22.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f16ffd700 1 -- 172.21.15.144:0/1062043327 >> v1:172.21.15.177:6789/0 conn(0x7f1f18161340 legacy=0x7f1f180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241742056 0 0) 0x7f1f000035b0 con 0x7f1f180cd7c0 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f16ffd700 1 -- 172.21.15.144:0/1062043327 >> v1:172.21.15.144:6789/0 conn(0x7f1f180cd7c0 legacy=0x7f1f180b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f16ffd700 1 -- 172.21.15.144:0/1062043327 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f180b5520 con 0x7f1f1815c9b0 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f16ffd700 1 --2- 172.21.15.144:0/1062043327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f180b4bf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4976166da99dd61a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100019f0 con 0x7f1f1815c9b0 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c004030 con 0x7f1f1815c9b0 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.195+0000 7f1f15ffb700 1 -- 172.21.15.144:0/1062043327 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c0043a0 con 0x7f1f1815c9b0 2023-11-28T11:51:22.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/1062043327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 msgr2=0x7f1f180b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/1062043327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f180b4bf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c009b80 comp rx=0 tx=0).stop 2023-11-28T11:51:22.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/1062043327 shutdown_connections 2023-11-28T11:51:22.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/1062043327 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f180b4bf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/1062043327 >> 172.21.15.144:0/1062043327 conn(0x7f1f18072600 msgr2=0x7f1f180cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/1062043327 shutdown_connections 2023-11-28T11:51:22.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.196+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/1062043327 wait complete. 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.197+0000 7f1f1d8cb700 1 Processor -- start 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.197+0000 7f1f1d8cb700 1 -- start start 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f1d8cb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f180cd7c0 0x7f1f1814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f1d8cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f1815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f1d8cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 0x7f1f1815e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f1d8cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181f6f50 con 0x7f1f18161340 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f1d8cb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181e6620 con 0x7f1f180cd7c0 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f1d8cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181e6900 con 0x7f1f1815c9b0 2023-11-28T11:51:22.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 0x7f1f1815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 0x7f1f1815e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34494/0 (socket says 172.21.15.144:34494) 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 -- 172.21.15.144:0/2732210368 learned_addr learned my addr 172.21.15.144:0/2732210368 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f167fc700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f1815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f16ffd700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f180cd7c0 0x7f1f1814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 -- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 msgr2=0x7f1f1815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f1815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 -- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f180cd7c0 msgr2=0x7f1f1814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f180cd7c0 0x7f1f1814a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 -- 172.21.15.144:0/2732210368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0094a0 con 0x7f1f18161340 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.198+0000 7f1f177fe700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 0x7f1f1815e440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000cba0 tx=0x7f1f1000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.199+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000aaa0 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.199+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1000ac20 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.199+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f180d15e0 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.199+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100070d0 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.199+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f180d1300 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.200+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f180b5130 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.200+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f1001a030 con 0x7f1f18161340 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.200+0000 7f1efffff700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f040682f0 0x7f1f0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.200+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f100858c0 con 0x7f1f18161340 2023-11-28T11:51:22.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.200+0000 7f1f16ffd700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f040682f0 0x7f1f0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.201+0000 7f1f16ffd700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f040682f0 0x7f1f0406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0097c0 tx=0x7f1f0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.205+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f10052120 con 0x7f1f18161340 2023-11-28T11:51:22.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.389+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f180745c0 con 0x7f1f18161340 2023-11-28T11:51:22.390 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:51:22.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.390+0000 7f1efffff700 1 -- 172.21.15.144:0/2732210368 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f1f10018090 con 0x7f1f18161340 2023-11-28T11:51:22.391 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:22.391 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:22.392 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:51:22.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f040682f0 msgr2=0x7f1f0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f040682f0 0x7f1f0406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0097c0 tx=0x7f1f0c008030 comp rx=0 tx=0).stop 2023-11-28T11:51:22.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 msgr2=0x7f1f1815e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 0x7f1f1815e440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000cba0 tx=0x7f1f1000ceb0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 shutdown_connections 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f18161340 0x7f1f1815e440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f040682f0 0x7f1f0406a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f180cd7c0 0x7f1f1814a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 --2- 172.21.15.144:0/2732210368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f1815c9b0 0x7f1f1815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.392+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 >> 172.21.15.144:0/2732210368 conn(0x7f1f18072600 msgr2=0x7f1f180b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.393+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 shutdown_connections 2023-11-28T11:51:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.393+0000 7f1f1d8cb700 1 -- 172.21.15.144:0/2732210368 wait complete. 2023-11-28T11:51:22.406 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T11:51:22.407 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T11:51:22.407 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T11:51:22.408 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T11:51:22.408 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T11:51:22.408 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T11:51:22.409 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T11:51:22.409 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T11:51:22.570 INFO:tasks.ceph.mds.b.smithi177.stderr:2023-11-28T11:51:22.569+0000 7f2efc871700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:51:22.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.637+0000 7fbb1b06e700 1 Processor -- start 2023-11-28T11:51:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.637+0000 7fbb1b06e700 1 -- start start 2023-11-28T11:51:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.638+0000 7fbb1b06e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbb1406d4f0 0x7fbb140b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.638+0000 7fbb1b06e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb140b1010 con 0x7fbb140b0bb0 2023-11-28T11:51:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.638+0000 7fbb1b06e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1415c490 con 0x7fbb14158890 2023-11-28T11:51:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.638+0000 7fbb1b06e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1415c730 con 0x7fbb1406d4f0 2023-11-28T11:51:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.638+0000 7fbb18e0a700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbb14158890 0x7fbb140ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51700/0 (socket says 172.21.15.144:51700) 2023-11-28T11:51:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.638+0000 7fbb18e0a700 1 -- 172.21.15.144:0/600663688 learned_addr learned my addr 172.21.15.144:0/600663688 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.640+0000 7fbb13fff700 1 --2- 172.21.15.144:0/600663688 >> v2:172.21.15.177:3300/0 conn(0x7fbb1406d4f0 0x7fbb140b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.642+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520487489 0 0) 0x7fbb1415c490 con 0x7fbb14158890 2023-11-28T11:51:22.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.642+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbafc0035b0 con 0x7fbb14158890 2023-11-28T11:51:22.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7f229d578700 1 Processor -- start 2023-11-28T11:51:22.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24190526 0 0) 0x7fbb140b1010 con 0x7fbb140b0bb0 2023-11-28T11:51:22.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb1415c490 con 0x7fbb140b0bb0 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 450828145 0 0) 0x7fbafc0035b0 con 0x7fbb14158890 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 >> v2:172.21.15.177:3300/0 conn(0x7fbb1406d4f0 msgr2=0x7fbb140b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 --2- 172.21.15.144:0/600663688 >> v2:172.21.15.177:3300/0 conn(0x7fbb1406d4f0 0x7fbb140b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 >> v1:172.21.15.144:6789/0 conn(0x7fbb140b0bb0 legacy=0x7fbb1406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1415ca10 con 0x7fbb14158890 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fbb0c001a30 con 0x7fbb14158890 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.643+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 >> v1:172.21.15.177:6789/0 conn(0x7fbb14158890 legacy=0x7fbb140ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb137fe700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbafc003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb137fe700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbafc004850 0x7fbafc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb137fe700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbafc005250 0x7fbafc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1415c730 con 0x7fbafc003be0 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc014970 con 0x7fbafc004850 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc014b30 con 0x7fbafc005250 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbafc003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb18e0a700 1 -- 172.21.15.144:0/600663688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbafc005250 msgr2=0x7fbafc00ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbafc005250 0x7fbafc00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb18e0a700 1 -- 172.21.15.144:0/600663688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbafc004850 msgr2=0x7fbafc004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbafc004850 0x7fbafc004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7fbb18e0a700 1 -- 172.21.15.144:0/600663688 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1415ca10 con 0x7fbafc003be0 2023-11-28T11:51:22.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbafc003fd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00a880 tx=0x7fbafc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6ded6de446488f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.644+0000 7f229d578700 1 -- start start 2023-11-28T11:51:22.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7fbb1960b700 1 -- 172.21.15.144:0/600663688 reap_dead start 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f229d578700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22980b0d80 0x7f2298163720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f229d578700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f229815c490 con 0x7f229806d4f0 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f229d578700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f229815c730 con 0x7f2298158890 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f229d578700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229815ca10 con 0x7f22980b0d80 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f22967fc700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f229806d4f0 0x7f22980b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51222/0 (socket says 172.21.15.144:51222) 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f22967fc700 1 -- 172.21.15.144:0/2644182740 learned_addr learned my addr 172.21.15.144:0/2644182740 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7f22977fe700 1 --2- 172.21.15.144:0/2644182740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22980b0d80 0x7f2298163720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c012600 con 0x7fbafc003be0 2023-11-28T11:51:22.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c012760 con 0x7fbafc003be0 2023-11-28T11:51:22.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.645+0000 7fbb137fe700 1 -- 172.21.15.144:0/600663688 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c012ad0 con 0x7fbafc003be0 2023-11-28T11:51:22.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.646+0000 7f22977fe700 1 -- 172.21.15.144:0/2644182740 >> v1:172.21.15.177:6789/0 conn(0x7f2298158890 legacy=0x7f22980ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.646+0000 7f22977fe700 1 -- 172.21.15.144:0/2644182740 >> v1:172.21.15.144:6789/0 conn(0x7f229806d4f0 legacy=0x7f22980b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.647+0000 7f22977fe700 1 -- 172.21.15.144:0/2644182740 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229815ccf0 con 0x7f22980b0d80 2023-11-28T11:51:22.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.647+0000 7f19485b9700 1 Processor -- start 2023-11-28T11:51:22.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.647+0000 7fbb1b06e700 1 -- 172.21.15.144:0/600663688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 msgr2=0x7fbafc003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.647+0000 7fbb1b06e700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbafc003fd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00a880 tx=0x7fbafc005880 comp rx=0 tx=0).stop 2023-11-28T11:51:22.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.647+0000 7f19485b9700 1 -- start start 2023-11-28T11:51:22.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7f22977fe700 1 --2- 172.21.15.144:0/2644182740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22980b0d80 0x7f2298163720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2290009b40 tx=0x7f229000d2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd4f30acdef4e98 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7fbb1b06e700 1 -- 172.21.15.144:0/600663688 shutdown_connections 2023-11-28T11:51:22.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7fbb1b06e700 1 --2- 172.21.15.144:0/600663688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbafc003fd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7fbb1b06e700 1 -- 172.21.15.144:0/600663688 >> 172.21.15.144:0/600663688 conn(0x7fbb140cb670 msgr2=0x7fbb140cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7fbb1b06e700 1 -- 172.21.15.144:0/600663688 shutdown_connections 2023-11-28T11:51:22.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7fbb1b06e700 1 -- 172.21.15.144:0/600663688 wait complete. 2023-11-28T11:51:22.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7f2295ffb700 1 -- 172.21.15.144:0/2644182740 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000dd80 con 0x7f22980b0d80 2023-11-28T11:51:22.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7f2295ffb700 1 -- 172.21.15.144:0/2644182740 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2290003a20 con 0x7f22980b0d80 2023-11-28T11:51:22.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.648+0000 7f2295ffb700 1 -- 172.21.15.144:0/2644182740 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000dee0 con 0x7f22980b0d80 2023-11-28T11:51:22.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f19485b9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f194006ba90 0x7f19400af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f19485b9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19400cbdf0 0x7f19400ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f19485b9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19400cc5f0 con 0x7f194015cef0 2023-11-28T11:51:22.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f19485b9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194015d470 con 0x7f194006ba90 2023-11-28T11:51:22.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f19485b9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194015d750 con 0x7f19400cbdf0 2023-11-28T11:51:22.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f229d578700 1 -- 172.21.15.144:0/2644182740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22980b0d80 msgr2=0x7f2298163720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f229d578700 1 --2- 172.21.15.144:0/2644182740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22980b0d80 0x7f2298163720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2290009b40 tx=0x7f229000d2f0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f1946b56700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19400cbdf0 0x7f19400ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f1945b54700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f194006ba90 0x7f19400af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f1945b54700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f194006ba90 0x7f19400af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34514/0 (socket says 172.21.15.144:34514) 2023-11-28T11:51:22.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.649+0000 7f1945b54700 1 -- 172.21.15.144:0/794346381 learned_addr learned my addr 172.21.15.144:0/794346381 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f229d578700 1 -- 172.21.15.144:0/2644182740 shutdown_connections 2023-11-28T11:51:22.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f229d578700 1 --2- 172.21.15.144:0/2644182740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22980b0d80 0x7f2298163720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f229d578700 1 -- 172.21.15.144:0/2644182740 >> 172.21.15.144:0/2644182740 conn(0x7f22980cb670 msgr2=0x7f22980cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945b54700 1 -- 172.21.15.144:0/794346381 >> v2:172.21.15.177:3300/0 conn(0x7f19400cbdf0 msgr2=0x7f19400ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945b54700 1 --2- 172.21.15.144:0/794346381 >> v2:172.21.15.177:3300/0 conn(0x7f19400cbdf0 0x7f19400ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945b54700 1 -- 172.21.15.144:0/794346381 >> v1:172.21.15.177:6789/0 conn(0x7f194015cef0 legacy=0x7f194006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945353700 1 -- 172.21.15.144:0/794346381 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950747020 0 0) 0x7f19400cc5f0 con 0x7f194015cef0 2023-11-28T11:51:22.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945b54700 1 -- 172.21.15.144:0/794346381 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194015da30 con 0x7f194006ba90 2023-11-28T11:51:22.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7fbb1b06e700 1 Processor -- start 2023-11-28T11:51:22.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f3102143700 1 Processor -- start 2023-11-28T11:51:22.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945b54700 1 --2- 172.21.15.144:0/794346381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194006ba90 0x7f19400af400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1940073b10 tx=0x7f193c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140d4980987456dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f229d578700 1 -- 172.21.15.144:0/2644182740 shutdown_connections 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945353700 1 -- 172.21.15.144:0/794346381 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c012b80 con 0x7f194006ba90 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945353700 1 -- 172.21.15.144:0/794346381 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c012ce0 con 0x7f194006ba90 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f1945353700 1 -- 172.21.15.144:0/794346381 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c01b620 con 0x7f194006ba90 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f19485b9700 1 -- 172.21.15.144:0/794346381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194006ba90 msgr2=0x7f19400af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f19485b9700 1 --2- 172.21.15.144:0/794346381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194006ba90 0x7f19400af400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1940073b10 tx=0x7f193c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7f3102143700 1 -- start start 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f3102143700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f30fc06e9f0 0x7f30fc06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f3102143700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc06ae60 con 0x7f30fc06d0a0 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f3102143700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc06a270 con 0x7f30fc06b830 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7f3102143700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc06a530 con 0x7f30fc06e9f0 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.650+0000 7fbb1b06e700 1 -- start start 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7fbb1b06e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbb141feae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7fbb1b06e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb141ff020 0x7fbb14151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7fbb1b06e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb14152050 0x7fbb141489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7fbb1b06e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb141524f0 con 0x7fbafc003be0 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7fbb1b06e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb14147960 con 0x7fbb141ff020 2023-11-28T11:51:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.651+0000 7fbb1b06e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb14158e50 con 0x7fbb14152050 2023-11-28T11:51:22.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30faffd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f30fc06b830 0x7f30fc06c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51726/0 (socket says 172.21.15.144:51726) 2023-11-28T11:51:22.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30faffd700 1 -- 172.21.15.144:0/3932140823 learned_addr learned my addr 172.21.15.144:0/3932140823 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb13fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb141ff020 0x7fbb14151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb13fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb141ff020 0x7fbb14151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36802/0 (socket says 172.21.15.144:36802) 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb13fff700 1 -- 172.21.15.144:0/360813939 learned_addr learned my addr 172.21.15.144:0/360813939 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb1960b700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb14152050 0x7fbb141489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbb141feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fb7fe700 1 --2- 172.21.15.144:0/3932140823 >> v2:172.21.15.144:3300/0 conn(0x7f30fc06e9f0 0x7f30fc06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f19485b9700 1 -- 172.21.15.144:0/794346381 shutdown_connections 2023-11-28T11:51:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f19485b9700 1 --2- 172.21.15.144:0/794346381 >> v2:172.21.15.177:3300/0 conn(0x7f19400cbdf0 0x7f19400ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f19485b9700 1 --2- 172.21.15.144:0/794346381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194006ba90 0x7f19400af400 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f19485b9700 1 -- 172.21.15.144:0/794346381 >> 172.21.15.144:0/794346381 conn(0x7f19400d0000 msgr2=0x7f19400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f229d578700 1 -- 172.21.15.144:0/2644182740 wait complete. 2023-11-28T11:51:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f3868cbb700 1 Processor -- start 2023-11-28T11:51:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f3868cbb700 1 -- start start 2023-11-28T11:51:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952256460 0 0) 0x7f30fc06ae60 con 0x7f30fc06d0a0 2023-11-28T11:51:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e40035b0 con 0x7f30fc06d0a0 2023-11-28T11:51:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb1960b700 1 -- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb141ff020 msgr2=0x7fbb14151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb1960b700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb141ff020 0x7fbb14151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb1960b700 1 -- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 msgr2=0x7fbb141feae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb1960b700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbafc003be0 0x7fbb141feae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7fbb1960b700 1 -- 172.21.15.144:0/360813939 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140c9370 con 0x7fbb14152050 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1437970251 0 0) 0x7f30e40035b0 con 0x7f30fc06d0a0 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 >> v2:172.21.15.144:3300/0 conn(0x7f30fc06e9f0 msgr2=0x7f30fc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 --2- 172.21.15.144:0/3932140823 >> v2:172.21.15.144:3300/0 conn(0x7f30fc06e9f0 0x7f30fc06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 >> v1:172.21.15.177:6789/0 conn(0x7f30fc06b830 legacy=0x7f30fc06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc06a810 con 0x7f30fc06d0a0 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.652+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f30f00015a0 con 0x7f30fc06d0a0 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 >> v1:172.21.15.144:6789/0 conn(0x7f30fc06d0a0 legacy=0x7f30fc0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e4003a00 0x7f30e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4004850 0x7f30e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4005200 0x7f30e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc06a530 con 0x7f30e4004850 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e400c190 con 0x7f30e4005200 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e400c350 con 0x7f30e4003a00 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4005200 0x7f30e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f19485b9700 1 -- 172.21.15.144:0/794346381 shutdown_connections 2023-11-28T11:51:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 -- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e4003a00 msgr2=0x7f30e4003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30e4003a00 0x7f30e4003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 -- 172.21.15.144:0/3932140823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4004850 msgr2=0x7f30e4004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4004850 0x7f30e4004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 -- 172.21.15.144:0/3932140823 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc06a810 con 0x7f30e4005200 2023-11-28T11:51:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f30fbfff700 1 -- 172.21.15.144:0/3932140823 reap_dead start 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f3868cbb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38640cdc30 0x7f38640b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f3868cbb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 0x7f386406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f3868cbb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38640cd7a0 con 0x7f3864159660 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f3868cbb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38640b1030 con 0x7f38640cdc30 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.653+0000 7f3868cbb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3864159a30 con 0x7f38640b0bb0 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 0x7f386406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 0x7f386406fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46618/0 (socket says 172.21.15.144:46618) 2023-11-28T11:51:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 -- 172.21.15.144:0/4198683338 learned_addr learned my addr 172.21.15.144:0/4198683338 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 -- 172.21.15.144:0/4198683338 >> v2:172.21.15.144:3300/0 conn(0x7f38640cdc30 msgr2=0x7f38640b0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 --2- 172.21.15.144:0/4198683338 >> v2:172.21.15.144:3300/0 conn(0x7f38640cdc30 0x7f38640b0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 -- 172.21.15.144:0/4198683338 >> v1:172.21.15.144:6789/0 conn(0x7f3864159660 legacy=0x7f38640c97b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f229d578700 1 Processor -- start 2023-11-28T11:51:22.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 -- 172.21.15.144:0/4198683338 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640ccb90 con 0x7f38640b0bb0 2023-11-28T11:51:22.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f3862d9d700 1 --2- 172.21.15.144:0/4198683338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 0x7f386406fb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38580098f0 tx=0x7f3858009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3c184ce9d0d90f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.655+0000 7f386159a700 1 -- 172.21.15.144:0/4198683338 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858004230 con 0x7f38640b0bb0 2023-11-28T11:51:22.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f19485b9700 1 -- 172.21.15.144:0/794346381 wait complete. 2023-11-28T11:51:22.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.655+0000 7f19485b9700 1 Processor -- start 2023-11-28T11:51:22.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.655+0000 7fbb1960b700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb14152050 0x7fbb141489c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb14068890 tx=0x7fbb0800c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f30fbfff700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4005200 0x7f30e400ba10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30f0007fc0 tx=0x7f30e4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe009672b92c2cec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08007310 con 0x7fbb14152050 2023-11-28T11:51:22.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb080074b0 con 0x7fbb14152050 2023-11-28T11:51:22.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08005850 con 0x7fbb14152050 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.654+0000 7f229d578700 1 -- start start 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f229d578700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f229806d4f0 0x7f22981f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f229d578700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22980b0d80 0x7f22980b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f229d578700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298158890 0x7f22980c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f229d578700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22982032c0 con 0x7f2298158890 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f229d578700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22982026b0 con 0x7f22980b0d80 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f229d578700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2298202990 con 0x7f229806d4f0 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f22967fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22980b0d80 0x7f22980b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f2296ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f229806d4f0 0x7f22981f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f22967fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22980b0d80 0x7f22980b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36816/0 (socket says 172.21.15.144:36816) 2023-11-28T11:51:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f22967fc700 1 -- 172.21.15.144:0/4103859557 learned_addr learned my addr 172.21.15.144:0/4103859557 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1415ccf0 con 0x7fbb14152050 2023-11-28T11:51:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb14158c60 con 0x7fbb14152050 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f22977fe700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298158890 0x7f22980c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f2296ffd700 1 -- 172.21.15.144:0/4103859557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22980b0d80 msgr2=0x7f22980b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f2296ffd700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22980b0d80 0x7f22980b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f2296ffd700 1 -- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298158890 msgr2=0x7f22980c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f2296ffd700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2298158890 0x7f22980c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f2296ffd700 1 -- 172.21.15.144:0/4103859557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290009770 con 0x7f229806d4f0 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.655+0000 7f19485b9700 1 -- start start 2023-11-28T11:51:22.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f19485b9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f194006ba90 0x7f19400af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f19485b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19400cbdf0 0x7f194015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f19485b9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f194015cef0 0x7f19400ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f19485b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19400ae950 con 0x7f19400cbdf0 2023-11-28T11:51:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f19485b9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19400add10 con 0x7f194015cef0 2023-11-28T11:51:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f19485b9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194006b220 con 0x7f194006ba90 2023-11-28T11:51:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3684bb2700 1 Processor -- start 2023-11-28T11:51:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19400cbdf0 0x7f194015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19400cbdf0 0x7f194015c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34550/0 (socket says 172.21.15.144:34550) 2023-11-28T11:51:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 -- 172.21.15.144:0/3374747174 learned_addr learned my addr 172.21.15.144:0/3374747174 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.655+0000 7f386159a700 1 -- 172.21.15.144:0/4198683338 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3858004390 con 0x7f38640b0bb0 2023-11-28T11:51:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.655+0000 7f386159a700 1 -- 172.21.15.144:0/4198683338 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38580036a0 con 0x7f38640b0bb0 2023-11-28T11:51:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000cc20 con 0x7f30e4005200 2023-11-28T11:51:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f3868cbb700 1 -- 172.21.15.144:0/4198683338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 msgr2=0x7f386406fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f3868cbb700 1 --2- 172.21.15.144:0/4198683338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 0x7f386406fb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38580098f0 tx=0x7f3858009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f000cd80 con 0x7f30e4005200 2023-11-28T11:51:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.656+0000 7f30fa7fc700 1 -- 172.21.15.144:0/3932140823 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00139b0 con 0x7f30e4005200 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3868cbb700 1 -- 172.21.15.144:0/4198683338 shutdown_connections 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3868cbb700 1 --2- 172.21.15.144:0/4198683338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38640b0bb0 0x7f386406fb30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3868cbb700 1 --2- 172.21.15.144:0/4198683338 >> v2:172.21.15.144:3300/0 conn(0x7f38640cdc30 0x7f38640b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3868cbb700 1 -- 172.21.15.144:0/4198683338 >> 172.21.15.144:0/4198683338 conn(0x7f38640cb670 msgr2=0x7f38640cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 -- 172.21.15.144:0/3374747174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f194006ba90 msgr2=0x7f19400af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f194006ba90 0x7f19400af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 -- 172.21.15.144:0/3374747174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f194015cef0 msgr2=0x7f19400ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f194015cef0 0x7f19400ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 -- 172.21.15.144:0/3374747174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c009550 con 0x7f19400cbdf0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f1945b54700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19400cbdf0 0x7f194015c210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f194006c7b0 tx=0x7f193c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c012510 con 0x7f19400cbdf0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c0036a0 con 0x7f19400cbdf0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194015dff0 con 0x7f19400cbdf0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194015dd10 con 0x7f19400cbdf0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c003a10 con 0x7f19400cbdf0 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3868cbb700 1 -- 172.21.15.144:0/4198683338 shutdown_connections 2023-11-28T11:51:22.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3868cbb700 1 -- 172.21.15.144:0/4198683338 wait complete. 2023-11-28T11:51:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3102143700 1 -- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4005200 msgr2=0x7f30e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3102143700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4005200 0x7f30e400ba10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30f0007fc0 tx=0x7f30e4005880 comp rx=0 tx=0).stop 2023-11-28T11:51:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3102143700 1 -- 172.21.15.144:0/3932140823 shutdown_connections 2023-11-28T11:51:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3102143700 1 --2- 172.21.15.144:0/3932140823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30e4005200 0x7f30e400ba10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3102143700 1 -- 172.21.15.144:0/3932140823 >> 172.21.15.144:0/3932140823 conn(0x7f30fc0b12b0 msgr2=0x7f30fc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb08010030 con 0x7fbb14152050 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7fbb117fa700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682f0 0x7fbb0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbb08083e70 con 0x7fbb14152050 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.657+0000 7f3684bb2700 1 -- start start 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3684bb2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 0x7f3680164770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3684bb2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f36800cb430 0x7f3680164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3684bb2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f36801651f0 0x7f36800cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3684bb2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800cc4b0 con 0x7f36801651f0 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3684bb2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800cedf0 con 0x7f36800cb430 2023-11-28T11:51:22.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.658+0000 7f3684bb2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800cb930 con 0x7f36800cea20 2023-11-28T11:51:22.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f2296ffd700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f229806d4f0 0x7f22981f8e10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f228c00b3c0 tx=0x7f228c00b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 0x7f3680164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7fbb1b06e700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fbaf40015e0 0x7fbaf4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 --> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbaf4006bc0 con 0x7fbaf40015e0 2023-11-28T11:51:22.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 0x7f3680164770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46634/0 (socket says 172.21.15.144:46634) 2023-11-28T11:51:22.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 -- 172.21.15.144:0/1456153379 learned_addr learned my addr 172.21.15.144:0/1456153379 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367ed9d700 1 --2- 172.21.15.144:0/1456153379 >> v2:172.21.15.144:3300/0 conn(0x7f36801651f0 0x7f36800cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011830 con 0x7f229806d4f0 2023-11-28T11:51:22.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682f0 0x7fbb0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c011990 con 0x7f229806d4f0 2023-11-28T11:51:22.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011d40 con 0x7f229806d4f0 2023-11-28T11:51:22.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2298158c60 con 0x7f229806d4f0 2023-11-28T11:51:22.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f193c021070 con 0x7f19400cbdf0 2023-11-28T11:51:22.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 -- 172.21.15.144:0/1456153379 >> v2:172.21.15.177:3300/0 conn(0x7f36800cb430 msgr2=0x7f3680164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 --2- 172.21.15.144:0/1456153379 >> v2:172.21.15.177:3300/0 conn(0x7f36800cb430 0x7f3680164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 -- 172.21.15.144:0/1456153379 >> v2:172.21.15.144:3300/0 conn(0x7f36801651f0 msgr2=0x7f36800cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 --2- 172.21.15.144:0/1456153379 >> v2:172.21.15.144:3300/0 conn(0x7f36801651f0 0x7f36800cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 -- 172.21.15.144:0/1456153379 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800cbc10 con 0x7f36800cea20 2023-11-28T11:51:22.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.659+0000 7f367e59c700 1 --2- 172.21.15.144:0/1456153379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 0x7f3680164770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36800d01a0 tx=0x7f367800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84cd436a6cd7500d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7fbb18e0a700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682f0 0x7fbb0006a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000f80 tx=0x7fbb0c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22980c9370 con 0x7f229806d4f0 2023-11-28T11:51:22.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3868cbb700 1 Processor -- start 2023-11-28T11:51:22.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3868cbb700 1 -- start start 2023-11-28T11:51:22.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7f367d59a700 1 -- 172.21.15.144:0/1456153379 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800e070 con 0x7f36800cea20 2023-11-28T11:51:22.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7f367d59a700 1 -- 172.21.15.144:0/1456153379 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800b910 con 0x7f36800cea20 2023-11-28T11:51:22.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7f367d59a700 1 -- 172.21.15.144:0/1456153379 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800bca0 con 0x7f36800cea20 2023-11-28T11:51:22.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 -- 172.21.15.144:0/1456153379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 msgr2=0x7f3680164770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 --2- 172.21.15.144:0/1456153379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 0x7f3680164770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36800d01a0 tx=0x7f367800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7f1936ffd700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19240682f0 0x7f192406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.660+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f193c0871c0 con 0x7f19400cbdf0 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f1946355700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19240682f0 0x7f192406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 -- 172.21.15.144:0/1456153379 shutdown_connections 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 --2- 172.21.15.144:0/1456153379 >> v2:172.21.15.144:3300/0 conn(0x7f36801651f0 0x7f36800cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 --2- 172.21.15.144:0/1456153379 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cea20 0x7f3680164770 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36800d01a0 tx=0x7f367800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:22.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 --2- 172.21.15.144:0/1456153379 >> v2:172.21.15.177:3300/0 conn(0x7f36800cb430 0x7f3680164cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f3684bb2700 1 -- 172.21.15.144:0/1456153379 >> 172.21.15.144:0/1456153379 conn(0x7f36800b0c30 msgr2=0x7f36800b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22980ce8f0 con 0x7f229806d4f0 2023-11-28T11:51:22.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f228c00f050 con 0x7f229806d4f0 2023-11-28T11:51:22.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f19485b9700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f19280015e0 0x7f1928003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.661+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1928006bc0 con 0x7f19280015e0 2023-11-28T11:51:22.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f1946355700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19240682f0 0x7f192406a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1930000f60 tx=0x7f1930007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3102143700 1 -- 172.21.15.144:0/3932140823 shutdown_connections 2023-11-28T11:51:22.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3684bb2700 1 -- 172.21.15.144:0/1456153379 shutdown_connections 2023-11-28T11:51:22.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3684bb2700 1 -- 172.21.15.144:0/1456153379 wait complete. 2023-11-28T11:51:22.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f227f7fe700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2284068340 0x7f228406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7fbb13fff700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fbaf40015e0 0x7fbaf4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7fac81c94700 1 Processor -- start 2023-11-28T11:51:22.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f228c08e3a0 con 0x7f229806d4f0 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f1946b56700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f19280015e0 0x7f1928003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3868cbb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38640b0bb0 0x7f3864071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3868cbb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38640cdc30 0x7f38640c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3868cbb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3864159660 0x7f38640c3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3868cbb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864158a90 con 0x7f38640cdc30 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3868cbb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864157e80 con 0x7f38640b0bb0 2023-11-28T11:51:22.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3868cbb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3864158160 con 0x7f3864159660 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7f3861d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38640cdc30 0x7f38640c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7f3861d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38640cdc30 0x7f38640c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34572/0 (socket says 172.21.15.144:34572) 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7f3861d9b700 1 -- 172.21.15.144:0/295475790 learned_addr learned my addr 172.21.15.144:0/295475790 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f3102143700 1 -- 172.21.15.144:0/3932140823 wait complete. 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f227f7fe700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7f2284070bf0 0x7f2284073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 --> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22840736b0 con 0x7f2284070bf0 2023-11-28T11:51:22.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f228c08e720 con 0x7f229806d4f0 2023-11-28T11:51:22.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f22977fe700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7f2284070bf0 0x7f2284073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7f22977fe700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7f2284070bf0 0x7f2284073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== osd.0 v2:172.21.15.144:6816/103558 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f22840736b0 con 0x7f2284070bf0 2023-11-28T11:51:22.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7f1946b56700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f19280015e0 0x7f1928003a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== osd.1 v2:172.21.15.144:6808/103557 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1928006bc0 con 0x7f19280015e0 2023-11-28T11:51:22.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7f386259c700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38640b0bb0 0x7f3864071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7f3862d9d700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3864159660 0x7f38640c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.662+0000 7fac81c94700 1 -- start start 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac81c94700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c06d4f0 0x7fac7c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac81c94700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fac7c0b0bb0 0x7fac7c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac81c94700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7c0b1000 con 0x7fac7c158890 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac81c94700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c15c490 con 0x7fac7c0b0bb0 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac81c94700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c15c730 con 0x7fac7c06d4f0 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac7b7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fac7c158890 0x7fac7c0ce110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51778/0 (socket says 172.21.15.144:51778) 2023-11-28T11:51:22.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.663+0000 7fac7b7fe700 1 -- 172.21.15.144:0/329087906 learned_addr learned my addr 172.21.15.144:0/329087906 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083710910 0 0) 0x7fac6c001490 con 0x7fac7c158890 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac680035b0 con 0x7fac7c158890 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2511421988 0 0) 0x7fac680035b0 con 0x7fac7c158890 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c06d4f0 msgr2=0x7fac7c0b0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c06d4f0 0x7fac7c0b0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 >> v2:172.21.15.144:3300/0 conn(0x7fac7c0b0bb0 msgr2=0x7fac7c06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 --2- 172.21.15.144:0/329087906 >> v2:172.21.15.144:3300/0 conn(0x7fac7c0b0bb0 0x7fac7c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c15ca10 con 0x7fac7c158890 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fac6c001b80 con 0x7fac7c158890 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 >> v1:172.21.15.177:6789/0 conn(0x7fac7c158890 legacy=0x7fac7c0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac680037f0 0x7fac68003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac68004140 0x7fac6800b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac6800b5e0 0x7fac6800da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c15c490 con 0x7fac68004140 2023-11-28T11:51:22.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c15c730 con 0x7fac6800b5e0 2023-11-28T11:51:22.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6800e380 con 0x7fac680037f0 2023-11-28T11:51:22.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7f3684bb2700 1 Processor -- start 2023-11-28T11:51:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3861d9b700 1 -- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3864159660 msgr2=0x7f38640c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3861d9b700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3864159660 0x7f38640c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3861d9b700 1 -- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38640b0bb0 msgr2=0x7f3864071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3861d9b700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38640b0bb0 0x7f3864071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3861d9b700 1 -- 172.21.15.144:0/295475790 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858009550 con 0x7f38640cdc30 2023-11-28T11:51:22.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3862d9d700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3864159660 0x7f38640c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:22.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7f3684bb2700 1 -- start start 2023-11-28T11:51:22.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3684bb2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36800cb430 0x7f36800c1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3684bb2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36801651f0 0x7f36800c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3684bb2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cd8c0 0x7f36800c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3684bb2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800be6a0 con 0x7f36800cb430 2023-11-28T11:51:22.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3684bb2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800bddb0 con 0x7f36801651f0 2023-11-28T11:51:22.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3684bb2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c4220 con 0x7f36800cd8c0 2023-11-28T11:51:22.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f367ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cd8c0 0x7f36800c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f367ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cd8c0 0x7f36800c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46668/0 (socket says 172.21.15.144:46668) 2023-11-28T11:51:22.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f367ed9d700 1 -- 172.21.15.144:0/605186968 learned_addr learned my addr 172.21.15.144:0/605186968 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f367e59c700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36800cb430 0x7f36800c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3861d9b700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38640cdc30 0x7f38640c32d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f385400b550 tx=0x7f385400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f22967fc700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2284068340 0x7f228406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367dd9b700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36801651f0 0x7f36800c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367e59c700 1 -- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cd8c0 msgr2=0x7f36800c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367e59c700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36800cd8c0 0x7f36800c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367e59c700 1 -- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36801651f0 msgr2=0x7f36800c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367e59c700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36801651f0 0x7f36800c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367e59c700 1 -- 172.21.15.144:0/605186968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678009b30 con 0x7f36800cb430 2023-11-28T11:51:22.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fbb13fff700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fbaf40015e0 0x7fbaf4003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f367e59c700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36800cb430 0x7f36800c1880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3678015000 tx=0x7f3678004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f22967fc700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2284068340 0x7f228406a7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2298151e60 tx=0x7f2288007e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7b7fe700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac680037f0 0x7fac68003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac6800b5e0 0x7fac6800da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 -- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac680037f0 msgr2=0x7fac68003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac680037f0 0x7fac68003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 -- 172.21.15.144:0/329087906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac68004140 msgr2=0x7fac6800b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac68004140 0x7fac6800b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 -- 172.21.15.144:0/329087906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c15ca10 con 0x7fac6800b5e0 2023-11-28T11:51:22.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7fac7bfff700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac6800b5e0 0x7fac6800da20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac74009910 tx=0x7fac68004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ddecf93a6a809a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74011620 con 0x7fac6800b5e0 2023-11-28T11:51:22.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678004450 con 0x7f36800cb430 2023-11-28T11:51:22.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36780045b0 con 0x7f36800cb430 2023-11-28T11:51:22.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678004960 con 0x7f36800cb430 2023-11-28T11:51:22.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011830 con 0x7f38640cdc30 2023-11-28T11:51:22.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38540119b0 con 0x7f38640cdc30 2023-11-28T11:51:22.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854011d60 con 0x7f38640cdc30 2023-11-28T11:51:22.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38640cd150 con 0x7f38640cdc30 2023-11-28T11:51:22.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36800cc1d0 con 0x7f36800cb430 2023-11-28T11:51:22.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38640cce70 con 0x7f38640cdc30 2023-11-28T11:51:22.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f38640cd430 con 0x7f38640cdc30 2023-11-28T11:51:22.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36800cbef0 con 0x7f36800cb430 2023-11-28T11:51:22.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.668+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3664000fc0 con 0x7f36800cb430 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7f3102143700 1 Processor -- start 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.664+0000 7f3102143700 1 -- start start 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3102143700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4005200 0x7f30fc0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3102143700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30fc158690 0x7f30fc1f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3102143700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30fc15aed0 0x7f30fc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3102143700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c8c90 con 0x7f30e4005200 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3102143700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc15a110 con 0x7f30fc15aed0 2023-11-28T11:51:22.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.665+0000 7f3102143700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc15a3f0 con 0x7f30fc158690 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30fc15aed0 0x7f30fc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30fc15aed0 0x7f30fc0b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36868/0 (socket says 172.21.15.144:36868) 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fbfff700 1 -- 172.21.15.144:0/3996569441 learned_addr learned my addr 172.21.15.144:0/3996569441 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30faffd700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30fc158690 0x7f30fc1f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fb7fe700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4005200 0x7f30fc0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fb7fe700 1 -- 172.21.15.144:0/3996569441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30fc158690 msgr2=0x7f30fc1f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fb7fe700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30fc158690 0x7f30fc1f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fb7fe700 1 -- 172.21.15.144:0/3996569441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30fc15aed0 msgr2=0x7f30fc0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fb7fe700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30fc15aed0 0x7f30fc0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.666+0000 7f30fb7fe700 1 -- 172.21.15.144:0/3996569441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc06edc0 con 0x7f30e4005200 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.667+0000 7f30fb7fe700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4005200 0x7f30fc0c86b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30f400a650 tx=0x7f30f4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.668+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400f040 con 0x7f30e4005200 2023-11-28T11:51:22.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.668+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f4003c40 con 0x7f30e4005200 2023-11-28T11:51:22.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.668+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc06aaf0 con 0x7f30e4005200 2023-11-28T11:51:22.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.668+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc06ef50 con 0x7f30e4005200 2023-11-28T11:51:22.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== osd.4 v2:172.21.15.177:6808/103772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbaf4006bc0 con 0x7fbaf40015e0 2023-11-28T11:51:22.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f367802c070 con 0x7f36800cb430 2023-11-28T11:51:22.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f400a2e0 con 0x7f30e4005200 2023-11-28T11:51:22.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f30fc15a280 con 0x7f30e4005200 2023-11-28T11:51:22.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f385400f050 con 0x7f38640cdc30 2023-11-28T11:51:22.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7f366effd700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f36600682f0 0x7f366006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.669+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3678087690 con 0x7f36800cb430 2023-11-28T11:51:22.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f366effd700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f3660070930 0x7f3660072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 --> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36600733f0 con 0x7f3660070930 2023-11-28T11:51:22.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3678053ef0 con 0x7f36800cb430 2023-11-28T11:51:22.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f3852ffd700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38440683c0 0x7f384406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3854085e00 con 0x7f38640cdc30 2023-11-28T11:51:22.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f3852ffd700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f3844070c70 0x7f3844073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 --> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3844073730 con 0x7f3844070c70 2023-11-28T11:51:22.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38540861b0 con 0x7f38640cdc30 2023-11-28T11:51:22.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f386259c700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38440683c0 0x7f384406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.670+0000 7f3862d9d700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f3844070c70 0x7f3844073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.671+0000 7f3862d9d700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f3844070c70 0x7f3844073080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.671+0000 7f386259c700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38440683c0 0x7f384406a870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f386414ebf0 tx=0x7f385c0094f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.671+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30f400d640 con 0x7f30e4005200 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.671+0000 7f30e3fff700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30e8068340 0x7f30e806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.671+0000 7f30faffd700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30e8068340 0x7f30e806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.671+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30f4086490 con 0x7f30e4005200 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.672+0000 7f30e3fff700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f30e8070bf0 0x7f30e8073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.672+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 --> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30e80736b0 con 0x7f30e8070bf0 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.672+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30f4086910 con 0x7f30e4005200 2023-11-28T11:51:22.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.672+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== osd.2 v2:172.21.15.144:6800/103559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3844073730 con 0x7f3844070c70 2023-11-28T11:51:22.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.672+0000 7f30faffd700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30e8068340 0x7f30e806a7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005910 tx=0x7f30ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.676+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac74011780 con 0x7fac6800b5e0 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.681+0000 7f367ed9d700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f3660070930 0x7f3660072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.681+0000 7f367dd9b700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f36600682f0 0x7f366006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.681+0000 7f367dd9b700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f36600682f0 0x7f366006a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3670006fd0 tx=0x7f3670008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.682+0000 7f367ed9d700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f3660070930 0x7f3660072d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.683+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== osd.6 v2:172.21.15.177:6800/103771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f36600733f0 con 0x7f3660070930 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.676+0000 7f30fbfff700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f30e8070bf0 0x7f30e8073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.677+0000 7f30fbfff700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f30e8070bf0 0x7f30e8073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.678+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== osd.3 v2:172.21.15.144:6823/103575 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f30e80736b0 con 0x7f30e8070bf0 2023-11-28T11:51:22.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.686+0000 7fac7affd700 1 -- 172.21.15.144:0/329087906 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74011b10 con 0x7fac6800b5e0 2023-11-28T11:51:22.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.686+0000 7fac81c94700 1 -- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac6800b5e0 msgr2=0x7fac6800da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.686+0000 7fac81c94700 1 --2- 172.21.15.144:0/329087906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac6800b5e0 0x7fac6800da20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac74009910 tx=0x7fac68004870 comp rx=0 tx=0).stop 2023-11-28T11:51:22.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.686+0000 7fac7bfff700 1 -- 172.21.15.144:0/329087906 reap_dead start 2023-11-28T11:51:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.687+0000 7fbe60f3c700 1 Processor -- start 2023-11-28T11:51:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.687+0000 7fbe60f3c700 1 -- start start 2023-11-28T11:51:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 --> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2298068240 con 0x7f2284070bf0 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe60f3c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe5c164d10 0x7fbe5c1650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe60f3c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbe5c0c2780 0x7fbe5c165620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe60f3c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe5c0cbaf0 con 0x7fbe5c0c2d40 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe60f3c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5c0ca6b0 con 0x7fbe5c164d10 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe60f3c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5c162630 con 0x7fbe5c0c2780 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe5ad9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbe5c0c2780 0x7fbe5c165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe59d9b700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbe5c0c2d40 0x7fbe5c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51812/0 (socket says 172.21.15.144:51812) 2023-11-28T11:51:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe5ad9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbe5c0c2780 0x7fbe5c165620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36886/0 (socket says 172.21.15.144:36886) 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.688+0000 7fbe59d9b700 1 -- 172.21.15.144:0/3364829036 learned_addr learned my addr 172.21.15.144:0/3364829036 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/3364829036 >> v2:172.21.15.144:3300/0 conn(0x7fbe5c164d10 msgr2=0x7fbe5c1650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5ad9d700 1 --2- 172.21.15.144:0/3364829036 >> v2:172.21.15.144:3300/0 conn(0x7fbe5c164d10 0x7fbe5c1650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/3364829036 >> v1:172.21.15.177:6789/0 conn(0x7fbe5c0c2d40 legacy=0x7fbe5c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5959a700 1 -- 172.21.15.144:0/3364829036 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804129333 0 0) 0x7fbe5c0cbaf0 con 0x7fbe5c0c2d40 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/3364829036 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c165c00 con 0x7fbe5c0c2780 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5ad9d700 1 --2- 172.21.15.144:0/3364829036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2780 0x7fbe5c165620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c1628b0 tx=0x7fbe5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67f8d8ac64d09a42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5959a700 1 -- 172.21.15.144:0/3364829036 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400e070 con 0x7fbe5c0c2780 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5959a700 1 -- 172.21.15.144:0/3364829036 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5400bbd0 con 0x7fbe5c0c2780 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.689+0000 7fbe5959a700 1 -- 172.21.15.144:0/3364829036 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54012430 con 0x7fbe5c0c2780 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 -- 172.21.15.144:0/3364829036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2780 msgr2=0x7fbe5c165620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/3364829036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2780 0x7fbe5c165620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c1628b0 tx=0x7fbe5400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 -- 172.21.15.144:0/3364829036 shutdown_connections 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/3364829036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2780 0x7fbe5c165620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/3364829036 >> v2:172.21.15.144:3300/0 conn(0x7fbe5c164d10 0x7fbe5c1650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 -- 172.21.15.144:0/3364829036 >> 172.21.15.144:0/3364829036 conn(0x7fbe5c0b12b0 msgr2=0x7fbe5c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 -- 172.21.15.144:0/3364829036 shutdown_connections 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.690+0000 7fbe60f3c700 1 -- 172.21.15.144:0/3364829036 wait complete. 2023-11-28T11:51:22.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.691+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 --> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbaf4005ca0 con 0x7fbaf40015e0 2023-11-28T11:51:22.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.691+0000 7fbb117fa700 1 -- 172.21.15.144:0/360813939 <== osd.4 v2:172.21.15.177:6808/103772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbaf4005ca0 con 0x7fbaf40015e0 2023-11-28T11:51:22.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fbaf40015e0 msgr2=0x7fbaf4003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fbaf40015e0 0x7fbaf4003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682f0 msgr2=0x7fbb0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbb000682f0 0x7fbb0006a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000f80 tx=0x7fbb0c010000 comp rx=0 tx=0).stop 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb14152050 msgr2=0x7fbb141489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 --2- 172.21.15.144:0/360813939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb14152050 0x7fbb141489c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb14068890 tx=0x7fbb0800c670 comp rx=0 tx=0).stop 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1960b700 1 -- 172.21.15.144:0/360813939 reap_dead start 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 shutdown_connections 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 >> 172.21.15.144:0/360813939 conn(0x7fbb140cb670 msgr2=0x7fbb14152f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 shutdown_connections 2023-11-28T11:51:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.692+0000 7fbb1b06e700 1 -- 172.21.15.144:0/360813939 wait complete. 2023-11-28T11:51:22.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.693+0000 7fbe60f3c700 1 Processor -- start 2023-11-28T11:51:22.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.693+0000 7fbe60f3c700 1 -- start start 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe60f3c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe5c0c2780 0x7fbe5c14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe60f3c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2d40 0x7fbe5c0c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe60f3c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5c164d10 0x7fbe5c1eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe60f3c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c14b3c0 con 0x7fbe5c0c2780 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe60f3c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c0c7180 con 0x7fbe5c0c2d40 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe60f3c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c14b800 con 0x7fbe5c164d10 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe5ad9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5c164d10 0x7fbe5c1eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe5ad9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5c164d10 0x7fbe5c1eacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46700/0 (socket says 172.21.15.144:46700) 2023-11-28T11:51:22.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.694+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/1313904791 learned_addr learned my addr 172.21.15.144:0/1313904791 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2d40 msgr2=0x7fbe5c0c6c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe5ad9d700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe5c0c2d40 0x7fbe5c0c6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe5c0c2780 msgr2=0x7fbe5c14ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe5ad9d700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe5c0c2780 0x7fbe5c14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/1313904791 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54009a70 con 0x7fbe5c164d10 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe5ad9d700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5c164d10 0x7fbe5c1eacc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c162770 tx=0x7fbe54004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54012430 con 0x7fbe5c164d10 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe54020070 con 0x7fbe5c164d10 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.695+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54004280 con 0x7fbe5c164d10 2023-11-28T11:51:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.696+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c0cc000 con 0x7fbe5c164d10 2023-11-28T11:51:22.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.696+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c0cbd20 con 0x7fbe5c164d10 2023-11-28T11:51:22.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.698+0000 7f227f7fe700 1 -- 172.21.15.144:0/4103859557 <== osd.0 v2:172.21.15.144:6816/103558 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2298068240 con 0x7f2284070bf0 2023-11-28T11:51:22.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.698+0000 7fac81c94700 1 -- 172.21.15.144:0/329087906 shutdown_connections 2023-11-28T11:51:22.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.698+0000 7fac81c94700 1 -- 172.21.15.144:0/329087906 >> 172.21.15.144:0/329087906 conn(0x7fac7c0cb670 msgr2=0x7fac7c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.698+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe5402e070 con 0x7fbe5c164d10 2023-11-28T11:51:22.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe4affd700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe3c0682f0 0x7fbe3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe54022070 con 0x7fbe5c164d10 2023-11-28T11:51:22.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe5a59c700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe3c0682f0 0x7fbe3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fac81c94700 1 -- 172.21.15.144:0/329087906 shutdown_connections 2023-11-28T11:51:22.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fac81c94700 1 -- 172.21.15.144:0/329087906 wait complete. 2023-11-28T11:51:22.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7fbe5c068470 0x7fbe5c14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe5a59c700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe3c0682f0 0x7fbe3c06a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe500097b0 tx=0x7fbe50006d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 --> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe5c066110 con 0x7fbe5c068470 2023-11-28T11:51:22.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.699+0000 7fbe59d9b700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7fbe5c068470 0x7fbe5c14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.700+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 --> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f30fc15a560 con 0x7f30e8070bf0 2023-11-28T11:51:22.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.700+0000 7fbe59d9b700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7fbe5c068470 0x7fbe5c14bd20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.700+0000 7f30e3fff700 1 -- 172.21.15.144:0/3996569441 <== osd.3 v2:172.21.15.144:6823/103575 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f30fc15a560 con 0x7f30e8070bf0 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== osd.7 v2:172.21.15.177:6816/103773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbe5c066110 con 0x7fbe5c068470 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7f2284070bf0 msgr2=0x7f2284073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f229d578700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7f2284070bf0 0x7f2284073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2284068340 msgr2=0x7f228406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f229d578700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2284068340 0x7f228406a7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2298151e60 tx=0x7f2288007e10 comp rx=0 tx=0).stop 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f229806d4f0 msgr2=0x7f22981f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f229d578700 1 --2- 172.21.15.144:0/4103859557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f229806d4f0 0x7f22981f8e10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f228c00b3c0 tx=0x7f228c00b780 comp rx=0 tx=0).stop 2023-11-28T11:51:22.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f22977fe700 1 -- 172.21.15.144:0/4103859557 reap_dead start 2023-11-28T11:51:22.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7fac81c94700 1 Processor -- start 2023-11-28T11:51:22.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7fac81c94700 1 -- start start 2023-11-28T11:51:22.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.701+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 --> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3864158e20 con 0x7f3844070c70 2023-11-28T11:51:22.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1928005ca0 con 0x7f19280015e0 2023-11-28T11:51:22.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac81c94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6800b5e0 0x7fac7c148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac81c94700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c153770 0x7fac7c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac81c94700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c48a0 0x7fac7c0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac81c94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c154120 con 0x7fac6800b5e0 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac81c94700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1491b0 con 0x7fac7c0c48a0 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac81c94700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c0c2610 con 0x7fac7c153770 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6800b5e0 0x7fac7c148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6800b5e0 0x7fac7c148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34624/0 (socket says 172.21.15.144:34624) 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 -- 172.21.15.144:0/3720318209 learned_addr learned my addr 172.21.15.144:0/3720318209 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:22.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7f3852ffd700 1 -- 172.21.15.144:0/295475790 <== osd.2 v2:172.21.15.144:6800/103559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3858004030 con 0x7f3844070c70 2023-11-28T11:51:22.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac72ffd700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c153770 0x7fac7c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 -- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c153770 msgr2=0x7fac7c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac7c153770 0x7fac7c153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 -- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c48a0 msgr2=0x7fac7c0c4d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:22.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac7c0c48a0 0x7fac7c0c4d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7fac7b7fe700 1 -- 172.21.15.144:0/3720318209 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c0c9370 con 0x7fac6800b5e0 2023-11-28T11:51:22.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.703+0000 7f1936ffd700 1 -- 172.21.15.144:0/3374747174 <== osd.1 v2:172.21.15.144:6808/103557 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1928005ca0 con 0x7f19280015e0 2023-11-28T11:51:22.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 shutdown_connections 2023-11-28T11:51:22.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 >> 172.21.15.144:0/4103859557 conn(0x7f22980cb670 msgr2=0x7f22980cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 shutdown_connections 2023-11-28T11:51:22.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f229d578700 1 -- 172.21.15.144:0/4103859557 wait complete. 2023-11-28T11:51:22.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7fac7b7fe700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6800b5e0 0x7fac7c148c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00e410 tx=0x7fac6c00e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f30e8070bf0 msgr2=0x7f30e8073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3102143700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f30e8070bf0 0x7f30e8073000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f19280015e0 msgr2=0x7f1928003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f19485b9700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f19280015e0 0x7f1928003a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19240682f0 msgr2=0x7f192406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f19485b9700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19240682f0 0x7f192406a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1930000f60 tx=0x7f1930007a40 comp rx=0 tx=0).stop 2023-11-28T11:51:22.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19400cbdf0 msgr2=0x7f194015c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f19485b9700 1 --2- 172.21.15.144:0/3374747174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19400cbdf0 0x7f194015c210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f194006c7b0 tx=0x7f193c004750 comp rx=0 tx=0).stop 2023-11-28T11:51:22.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f3844070c70 msgr2=0x7f3844073080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3868cbb700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f3844070c70 0x7f3844073080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f1946b56700 1 -- 172.21.15.144:0/3374747174 reap_dead start 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38440683c0 msgr2=0x7f384406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3868cbb700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38440683c0 0x7f384406a870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f386414ebf0 tx=0x7f385c0094f0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38640cdc30 msgr2=0x7f38640c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7f3868cbb700 1 --2- 172.21.15.144:0/295475790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38640cdc30 0x7f38640c32d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f385400b550 tx=0x7f385400b860 comp rx=0 tx=0).stop 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.705+0000 7f3862d9d700 1 -- 172.21.15.144:0/295475790 reap_dead start 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.705+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 shutdown_connections 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.705+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 >> 172.21.15.144:0/295475790 conn(0x7f38640cb670 msgr2=0x7f386406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.705+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 shutdown_connections 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.705+0000 7f3868cbb700 1 -- 172.21.15.144:0/295475790 wait complete. 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 shutdown_connections 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 >> 172.21.15.144:0/3374747174 conn(0x7f19400d0000 msgr2=0x7f19400cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c013630 con 0x7fac6800b5e0 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c013790 con 0x7fac6800b5e0 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c013b40 con 0x7fac6800b5e0 2023-11-28T11:51:22.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c15ccf0 con 0x7fac6800b5e0 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.704+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c158c60 con 0x7fac6800b5e0 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30e8068340 msgr2=0x7f30e806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f3102143700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30e8068340 0x7f30e806a7f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005910 tx=0x7f30ec009040 comp rx=0 tx=0).stop 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4005200 msgr2=0x7f30fc0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f3102143700 1 --2- 172.21.15.144:0/3996569441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30e4005200 0x7f30fc0c86b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30f400a650 tx=0x7f30f4007ef0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f30fbfff700 1 -- 172.21.15.144:0/3996569441 reap_dead start 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 shutdown_connections 2023-11-28T11:51:22.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 >> 172.21.15.144:0/3996569441 conn(0x7f30fc0b12b0 msgr2=0x7f30fc0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 shutdown_connections 2023-11-28T11:51:22.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.706+0000 7f19485b9700 1 -- 172.21.15.144:0/3374747174 wait complete. 2023-11-28T11:51:22.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.707+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 shutdown_connections 2023-11-28T11:51:22.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.707+0000 7f3102143700 1 -- 172.21.15.144:0/3996569441 wait complete. 2023-11-28T11:51:22.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.708+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac60000fc0 con 0x7fac6800b5e0 2023-11-28T11:51:22.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.708+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac6c023070 con 0x7fac6800b5e0 2023-11-28T11:51:22.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.708+0000 7fac78ff9700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fac5c0683c0 0x7fac5c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.708+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac6c086840 con 0x7fac6800b5e0 2023-11-28T11:51:22.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.709+0000 7fac72ffd700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fac5c0683c0 0x7fac5c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.709+0000 7fac78ff9700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7fac5c070bf0 0x7fac5c073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.709+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac5c0736b0 con 0x7fac5c070bf0 2023-11-28T11:51:22.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.709+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac6c053030 con 0x7fac6800b5e0 2023-11-28T11:51:22.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.709+0000 7fac7bfff700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7fac5c070bf0 0x7fac5c073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.709+0000 7fac72ffd700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fac5c0683c0 0x7fac5c06a870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0c87f0 tx=0x7fac64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.748 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.748 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:51:22.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.711+0000 7fac7bfff700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7fac5c070bf0 0x7fac5c073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.712+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== osd.5 v2:172.21.15.177:6817/103774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fac5c0736b0 con 0x7fac5c070bf0 2023-11-28T11:51:22.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.721+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 --> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3664002d40 con 0x7f3660070930 2023-11-28T11:51:22.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.721+0000 7f366effd700 1 -- 172.21.15.144:0/605186968 <== osd.6 v2:172.21.15.177:6800/103771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3664002d40 con 0x7f3660070930 2023-11-28T11:51:22.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f3660070930 msgr2=0x7f3660072d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f3660070930 0x7f3660072d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f36600682f0 msgr2=0x7f366006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f36600682f0 0x7f366006a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3670006fd0 tx=0x7f3670008040 comp rx=0 tx=0).stop 2023-11-28T11:51:22.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36800cb430 msgr2=0x7f36800c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 --2- 172.21.15.144:0/605186968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36800cb430 0x7f36800c1880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3678015000 tx=0x7f3678004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f367ed9d700 1 -- 172.21.15.144:0/605186968 reap_dead start 2023-11-28T11:51:22.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 shutdown_connections 2023-11-28T11:51:22.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 >> 172.21.15.144:0/605186968 conn(0x7f36800b0c30 msgr2=0x7f36800cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.722+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 shutdown_connections 2023-11-28T11:51:22.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.723+0000 7f3684bb2700 1 -- 172.21.15.144:0/605186968 wait complete. 2023-11-28T11:51:22.752 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.752 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:51:22.753 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:51:22.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.729+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 --> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe5c0b5740 con 0x7fbe5c068470 2023-11-28T11:51:22.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe4affd700 1 -- 172.21.15.144:0/1313904791 <== osd.7 v2:172.21.15.177:6816/103773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe5c0b5740 con 0x7fbe5c068470 2023-11-28T11:51:22.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7fbe5c068470 msgr2=0x7fbe5c14bd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7fbe5c068470 0x7fbe5c14bd20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe3c0682f0 msgr2=0x7fbe3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe3c0682f0 0x7fbe3c06a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe500097b0 tx=0x7fbe50006d20 comp rx=0 tx=0).stop 2023-11-28T11:51:22.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5c164d10 msgr2=0x7fbe5c1eacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 --2- 172.21.15.144:0/1313904791 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe5c164d10 0x7fbe5c1eacc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c162770 tx=0x7fbe54004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe5ad9d700 1 -- 172.21.15.144:0/1313904791 reap_dead start 2023-11-28T11:51:22.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.730+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 shutdown_connections 2023-11-28T11:51:22.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.731+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 >> 172.21.15.144:0/1313904791 conn(0x7fbe5c0b12b0 msgr2=0x7fbe5c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.731+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 shutdown_connections 2023-11-28T11:51:22.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.731+0000 7fbe60f3c700 1 -- 172.21.15.144:0/1313904791 wait complete. 2023-11-28T11:51:22.756 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.757 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:51:22.758 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.758 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:51:22.759 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:22.759 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:51:22.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.738+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac60002d40 con 0x7fac5c070bf0 2023-11-28T11:51:22.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.738+0000 7fac78ff9700 1 -- 172.21.15.144:0/3720318209 <== osd.5 v2:172.21.15.177:6817/103774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac60002d40 con 0x7fac5c070bf0 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7fac5c070bf0 msgr2=0x7fac5c073000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7fac5c070bf0 0x7fac5c073000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fac5c0683c0 msgr2=0x7fac5c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fac5c0683c0 0x7fac5c06a870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0c87f0 tx=0x7fac64008040 comp rx=0 tx=0).stop 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6800b5e0 msgr2=0x7fac7c148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 --2- 172.21.15.144:0/3720318209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6800b5e0 0x7fac7c148c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00e410 tx=0x7fac6c00e7d0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac7bfff700 1 -- 172.21.15.144:0/3720318209 reap_dead start 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 shutdown_connections 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 >> 172.21.15.144:0/3720318209 conn(0x7fac7c0cb670 msgr2=0x7fac7c154910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 shutdown_connections 2023-11-28T11:51:22.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.739+0000 7fac81c94700 1 -- 172.21.15.144:0/3720318209 wait complete. 2023-11-28T11:51:22.762 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.763 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:51:22.763 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:22.763 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:51:22.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.988+0000 7fd6439fe700 1 Processor -- start 2023-11-28T11:51:22.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.988+0000 7fd6439fe700 1 -- start start 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.989+0000 7fd6439fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd63c0b0d80 0x7fd63c0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.989+0000 7fd6439fe700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c06ba50 con 0x7fd63c160b30 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.989+0000 7fd6439fe700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c164f90 con 0x7fd63c160f00 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.989+0000 7fd6439fe700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c165270 con 0x7fd63c0b0d80 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.990+0000 7fd640f99700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd63c160f00 0x7fd63c0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51838/0 (socket says 172.21.15.144:51838) 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.990+0000 7fd640f99700 1 -- 172.21.15.144:0/2749927152 learned_addr learned my addr 172.21.15.144:0/2749927152 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:22.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.991+0000 7fd641f9b700 1 --2- 172.21.15.144:0/2749927152 >> v2:172.21.15.144:3300/0 conn(0x7fd63c0b0d80 0x7fd63c0c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:22.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.991+0000 7fd641f9b700 1 -- 172.21.15.144:0/2749927152 >> v1:172.21.15.177:6789/0 conn(0x7fd63c160f00 legacy=0x7fd63c0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.991+0000 7fd641f9b700 1 -- 172.21.15.144:0/2749927152 >> v1:172.21.15.144:6789/0 conn(0x7fd63c160b30 legacy=0x7fd63c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.991+0000 7fd641f9b700 1 -- 172.21.15.144:0/2749927152 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c165550 con 0x7fd63c0b0d80 2023-11-28T11:51:22.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.991+0000 7fd641f9b700 1 --2- 172.21.15.144:0/2749927152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0b0d80 0x7fd63c0c1e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd634009d90 tx=0x7fd63400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae51b8b33525bcbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:22.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.993+0000 7fd633fff700 1 -- 172.21.15.144:0/2749927152 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400e070 con 0x7fd63c0b0d80 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.993+0000 7fd633fff700 1 -- 172.21.15.144:0/2749927152 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63400bad0 con 0x7fd63c0b0d80 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.993+0000 7fd633fff700 1 -- 172.21.15.144:0/2749927152 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400be80 con 0x7fd63c0b0d80 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.994+0000 7fd6439fe700 1 -- 172.21.15.144:0/2749927152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0b0d80 msgr2=0x7fd63c0c1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.994+0000 7fd6439fe700 1 --2- 172.21.15.144:0/2749927152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0b0d80 0x7fd63c0c1e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd634009d90 tx=0x7fd63400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.995+0000 7fd6439fe700 1 -- 172.21.15.144:0/2749927152 shutdown_connections 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.995+0000 7fd6439fe700 1 --2- 172.21.15.144:0/2749927152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0b0d80 0x7fd63c0c1e30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:22.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.995+0000 7fd6439fe700 1 -- 172.21.15.144:0/2749927152 >> 172.21.15.144:0/2749927152 conn(0x7fd63c0cb670 msgr2=0x7fd63c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:22.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.999+0000 7fd6439fe700 1 -- 172.21.15.144:0/2749927152 shutdown_connections 2023-11-28T11:51:22.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:22.999+0000 7fd6439fe700 1 -- 172.21.15.144:0/2749927152 wait complete. 2023-11-28T11:51:23.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.000+0000 7fe781111700 1 Processor -- start 2023-11-28T11:51:23.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.000+0000 7fd6439fe700 1 Processor -- start 2023-11-28T11:51:23.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.000+0000 7fd6439fe700 1 -- start start 2023-11-28T11:51:23.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.000+0000 7fe781111700 1 -- start start 2023-11-28T11:51:23.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd6439fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0b0d80 0x7fd63c140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd6439fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 0x7fd63c1411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd6439fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c160f00 0x7fd63c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd6439fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1456a0 con 0x7fd63c160b30 2023-11-28T11:51:23.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd6439fe700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c160680 con 0x7fd63c160f00 2023-11-28T11:51:23.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd6439fe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0b0bd0 con 0x7fd63c0b0d80 2023-11-28T11:51:23.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd640f99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 0x7fd63c1411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd640f99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 0x7fd63c1411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34648/0 (socket says 172.21.15.144:34648) 2023-11-28T11:51:23.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd640f99700 1 -- 172.21.15.144:0/356907878 learned_addr learned my addr 172.21.15.144:0/356907878 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd641f9b700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c160f00 0x7fd63c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fd64179a700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0b0d80 0x7fd63c140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe781111700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c158890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe781111700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe77c0b0670 0x7fe77c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe781111700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77c15c490 con 0x7fe77c06d4f0 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe781111700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77c0b1030 con 0x7fe77c0b0670 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe781111700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c15c730 con 0x7fe77c0cc180 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe77b59e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe77c0b0670 0x7fe77c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe77b59e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe77c0b0670 0x7fe77c0b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36916/0 (socket says 172.21.15.144:36916) 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.001+0000 7fe77b59e700 1 -- 172.21.15.144:0/3331255045 learned_addr learned my addr 172.21.15.144:0/3331255045 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fe77a59c700 1 --2- 172.21.15.144:0/3331255045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fe77a59c700 1 -- 172.21.15.144:0/3331255045 >> v2:172.21.15.177:3300/0 conn(0x7fe77c0b0670 msgr2=0x7fe77c0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fe77a59c700 1 --2- 172.21.15.144:0/3331255045 >> v2:172.21.15.177:3300/0 conn(0x7fe77c0b0670 0x7fe77c0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fe77a59c700 1 -- 172.21.15.144:0/3331255045 >> v1:172.21.15.177:6789/0 conn(0x7fe77c06d4f0 legacy=0x7fe77c0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.002+0000 7fe77a59c700 1 -- 172.21.15.144:0/3331255045 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c15ca10 con 0x7fe77c0cc180 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd640f99700 1 -- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0b0d80 msgr2=0x7fd63c140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd640f99700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0b0d80 0x7fd63c140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd640f99700 1 -- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c160f00 msgr2=0x7fd63c14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd640f99700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c160f00 0x7fd63c14f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd640f99700 1 -- 172.21.15.144:0/356907878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634009a70 con 0x7fd63c160b30 2023-11-28T11:51:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd640f99700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 0x7fd63c1411b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c480 tx=0x7fd63800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fe0cb843700 1 Processor -- start 2023-11-28T11:51:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fe0cb843700 1 -- start start 2023-11-28T11:51:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fe0cb843700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b51e0 0x7fe0c409c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fe0cb843700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0c40b5ea0 0x7fe0c406ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fe0cb843700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40b7dc0 con 0x7fe0c4160dc0 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fe0cb843700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40b0ea0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fe0cb843700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b1140 con 0x7fe0c40b51e0 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.003+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638005d70 con 0x7fd63c160b30 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63800ce90 con 0x7fd63c160b30 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380072d0 con 0x7fd63c160b30 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c164840 con 0x7fd63c160b30 2023-11-28T11:51:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.004+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c0cce40 con 0x7fd63c160b30 2023-11-28T11:51:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0c40b5ea0 0x7fe0c406ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe0c40b5ea0 0x7fe0c406ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36926/0 (socket says 172.21.15.144:36926) 2023-11-28T11:51:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 -- 172.21.15.144:0/1426222926 learned_addr learned my addr 172.21.15.144:0/1426222926 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c1649d0 con 0x7fd63c160b30 2023-11-28T11:51:23.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 -- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b51e0 msgr2=0x7fe0c409c4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 --2- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b51e0 0x7fe0c409c4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 -- 172.21.15.144:0/1426222926 >> v1:172.21.15.144:6789/0 conn(0x7fe0c4160dc0 legacy=0x7fe0c40b4ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.005+0000 7fe0c9de0700 1 -- 172.21.15.144:0/1426222926 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c409bab0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe0c9de0700 1 --2- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40b5ea0 0x7fe0c406ab30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40098f0 tx=0x7fe0b4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=242149b0fa558886 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe77a59c700 1 --2- 172.21.15.144:0/3331255045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c158890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098a0 tx=0x7fe770009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0218d7d951ce0a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe0bbfff700 1 -- 172.21.15.144:0/1426222926 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4004230 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe0bbfff700 1 -- 172.21.15.144:0/1426222926 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4004390 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe0bbfff700 1 -- 172.21.15.144:0/1426222926 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40036a0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe779d9b700 1 -- 172.21.15.144:0/3331255045 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770004030 con 0x7fe77c0cc180 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe779d9b700 1 -- 172.21.15.144:0/3331255045 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770004190 con 0x7fe77c0cc180 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.006+0000 7fe779d9b700 1 -- 172.21.15.144:0/3331255045 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770004540 con 0x7fe77c0cc180 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.007+0000 7fe781111700 1 -- 172.21.15.144:0/3331255045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 msgr2=0x7fe77c158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.007+0000 7fe781111700 1 --2- 172.21.15.144:0/3331255045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c158890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098a0 tx=0x7fe770009bb0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.007+0000 7fe0cb843700 1 -- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40b5ea0 msgr2=0x7fe0c406ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.007+0000 7fe0cb843700 1 --2- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40b5ea0 0x7fe0c406ab30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40098f0 tx=0x7fe0b4009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:23.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.007+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd638019030 con 0x7fd63c160b30 2023-11-28T11:51:23.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.007+0000 7fd631ffb700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200683c0 0x7fd62006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd638084e30 con 0x7fd63c160b30 2023-11-28T11:51:23.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe0cb843700 1 -- 172.21.15.144:0/1426222926 shutdown_connections 2023-11-28T11:51:23.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe0cb843700 1 --2- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40b5ea0 0x7fe0c406ab30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe0cb843700 1 --2- 172.21.15.144:0/1426222926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b51e0 0x7fe0c409c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe0cb843700 1 -- 172.21.15.144:0/1426222926 >> 172.21.15.144:0/1426222926 conn(0x7fe0c40cfad0 msgr2=0x7fe0c40cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe781111700 1 -- 172.21.15.144:0/3331255045 shutdown_connections 2023-11-28T11:51:23.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe781111700 1 --2- 172.21.15.144:0/3331255045 >> v2:172.21.15.177:3300/0 conn(0x7fe77c0b0670 0x7fe77c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe781111700 1 --2- 172.21.15.144:0/3331255045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c158890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe781111700 1 -- 172.21.15.144:0/3331255045 >> 172.21.15.144:0/3331255045 conn(0x7fe77c0cb670 msgr2=0x7fe77c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe781111700 1 -- 172.21.15.144:0/3331255045 shutdown_connections 2023-11-28T11:51:23.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fd64179a700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200683c0 0x7fd62006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.009+0000 7fd64179a700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200683c0 0x7fd62006a870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd62c007fd0 tx=0x7fd62c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe0cb843700 1 -- 172.21.15.144:0/1426222926 shutdown_connections 2023-11-28T11:51:23.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.009+0000 7fe0cb843700 1 -- 172.21.15.144:0/1426222926 wait complete. 2023-11-28T11:51:23.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.010+0000 7fe0cb843700 1 Processor -- start 2023-11-28T11:51:23.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.008+0000 7fe781111700 1 -- 172.21.15.144:0/3331255045 wait complete. 2023-11-28T11:51:23.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.010+0000 7fe781111700 1 Processor -- start 2023-11-28T11:51:23.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.010+0000 7fe781111700 1 -- start start 2023-11-28T11:51:23.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.010+0000 7fe0cb843700 1 -- start start 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe781111700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe77c06d4f0 0x7fe77c146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe781111700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 0x7fe77c154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe781111700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe781111700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0b24f0 con 0x7fe77c0b0670 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe781111700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1466d0 con 0x7fe77c06d4f0 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe781111700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0b1bc0 con 0x7fe77c0cc180 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe77a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 0x7fe77c154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe77a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 0x7fe77c154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34658/0 (socket says 172.21.15.144:34658) 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.011+0000 7fe77a59c700 1 -- 172.21.15.144:0/1221131408 learned_addr learned my addr 172.21.15.144:0/1221131408 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77ad9d700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe77c06d4f0 0x7fe77c146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0cb843700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c40b51e0 0x7fe0c40c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0cb843700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b5ea0 0x7fe0c40c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0cb843700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 0x7fe0c41fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0cb843700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c81a0 con 0x7fe0c40b51e0 2023-11-28T11:51:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0cb843700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c7590 con 0x7fe0c40c96b0 2023-11-28T11:51:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0cb843700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c7870 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c9de0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 0x7fe0c41fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c9de0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 0x7fe0c41fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36936/0 (socket says 172.21.15.144:36936) 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c9de0700 1 -- 172.21.15.144:0/3995735075 learned_addr learned my addr 172.21.15.144:0/3995735075 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c8dde700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b5ea0 0x7fe0c40c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c95df700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c40b51e0 0x7fe0c40c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77a59c700 1 -- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 msgr2=0x7fe77c070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77a59c700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77a59c700 1 -- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe77c06d4f0 msgr2=0x7fe77c146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77a59c700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe77c06d4f0 0x7fe77c146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77a59c700 1 -- 172.21.15.144:0/1221131408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700094a0 con 0x7fe77c0b0670 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c8dde700 1 -- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 msgr2=0x7fe0c41fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c8dde700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 0x7fe0c41fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c8dde700 1 -- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c40b51e0 msgr2=0x7fe0c40c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c8dde700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c40b51e0 0x7fe0c40c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c8dde700 1 -- 172.21.15.144:0/3995735075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4009550 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe0c9de0700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 0x7fe0c41fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.012+0000 7fe77a59c700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 0x7fe77c154e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe770009fd0 tx=0x7fe7700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.013+0000 7fe0c8dde700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b5ea0 0x7fe0c40c8630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00bdf0 tx=0x7fe0bc009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7f698567e700 1 Processor -- start 2023-11-28T11:51:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd63804b5e0 con 0x7fd63c160b30 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770003e00 con 0x7fe77c0b0670 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770022850 con 0x7fe77c0b0670 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770022c00 con 0x7fe77c0b0670 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.018+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c0c9370 con 0x7fe77c0b0670 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.018+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c15ccf0 con 0x7fe77c0b0670 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00cc90 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00cdf0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc014660 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c4161190 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c40b1420 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.018+0000 7f6a93493700 1 Processor -- start 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.018+0000 7f6a93493700 1 -- start start 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f6a93493700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6a8c0c2d40 0x7f6a8c161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f6a93493700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f6a93493700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c0cbaf0 con 0x7f6a8c164d10 2023-11-28T11:51:23.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f6a93493700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c0ca6b0 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f6a93493700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c165c00 con 0x7f6a8c0c2780 2023-11-28T11:51:23.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a90a2e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6a8c0c2d40 0x7f6a8c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46756/0 (socket says 172.21.15.144:46756) 2023-11-28T11:51:23.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 -- 172.21.15.144:0/311685289 learned_addr learned my addr 172.21.15.144:0/311685289 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.017+0000 7f698567e700 1 -- start start 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.018+0000 7f698567e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69800ce480 0x7f69800c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f698567e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6980164e60 0x7f69801652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f698567e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6980162210 con 0x7f6980161220 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f698567e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69801649d0 con 0x7f6980164e60 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.019+0000 7f698567e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698006c020 con 0x7f69800ce480 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f697f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6980164e60 0x7f69801652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f697f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6980164e60 0x7f69801652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34678/0 (socket says 172.21.15.144:34678) 2023-11-28T11:51:23.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f697f7fe700 1 -- 172.21.15.144:0/124663263 learned_addr learned my addr 172.21.15.144:0/124663263 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 -- 172.21.15.144:0/311685289 >> v2:172.21.15.177:3300/0 conn(0x7f6a8c0c2d40 msgr2=0x7f6a8c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:23.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 --2- 172.21.15.144:0/311685289 >> v2:172.21.15.177:3300/0 conn(0x7f6a8c0c2d40 0x7f6a8c161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 -- 172.21.15.144:0/311685289 >> v1:172.21.15.144:6789/0 conn(0x7f6a8c164d10 legacy=0x7f6a8c1650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:23.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f6a91a30700 1 -- 172.21.15.144:0/311685289 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c0cbd20 con 0x7f6a8c0c2780 2023-11-28T11:51:23.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0bc01c360 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f697f7fe700 1 -- 172.21.15.144:0/124663263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69800ce480 msgr2=0x7f69800c1fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:23.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f697f7fe700 1 --2- 172.21.15.144:0/124663263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69800ce480 0x7f69800c1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f697f7fe700 1 -- 172.21.15.144:0/124663263 >> v1:172.21.15.144:6789/0 conn(0x7f6980161220 legacy=0x7f698006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:23.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f697f7fe700 1 -- 172.21.15.144:0/124663263 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980161970 con 0x7f6980164e60 2023-11-28T11:51:23.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f697f7fe700 1 --2- 172.21.15.144:0/124663263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6980164e60 0x7f69801652e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f69740098f0 tx=0x7f6974009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd97dd6a246d82d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7fe0b9ffb700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe0a80683c0 0x7fe0a806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f32e9069700 1 Processor -- start 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f32e9069700 1 -- start start 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e9069700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f32e406ba90 0x7f32e40af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e9069700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 0x7f32e40ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e9069700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40cc5f0 con 0x7f32e415cef0 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e9069700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e415d470 con 0x7f32e406ba90 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e9069700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e415d750 con 0x7f32e40cbdf0 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e359e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 0x7f32e40ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e359e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 0x7f32e40ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46768/0 (socket says 172.21.15.144:46768) 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f32e359e700 1 -- 172.21.15.144:0/3554553183 learned_addr learned my addr 172.21.15.144:0/3554553183 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e259c700 1 --2- 172.21.15.144:0/3554553183 >> v2:172.21.15.177:3300/0 conn(0x7f32e406ba90 0x7f32e40af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0bc085a30 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7fe0c95df700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe0a80683c0 0x7fe0a806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e359e700 1 -- 172.21.15.144:0/3554553183 >> v2:172.21.15.177:3300/0 conn(0x7f32e406ba90 msgr2=0x7f32e40af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e359e700 1 --2- 172.21.15.144:0/3554553183 >> v2:172.21.15.177:3300/0 conn(0x7f32e406ba90 0x7f32e40af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e359e700 1 -- 172.21.15.144:0/3554553183 >> v1:172.21.15.144:6789/0 conn(0x7f32e415cef0 legacy=0x7f32e406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e359e700 1 -- 172.21.15.144:0/3554553183 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e415da30 con 0x7f32e40cbdf0 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e359e700 1 --2- 172.21.15.144:0/3554553183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 0x7f32e40ca650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009e50 tx=0x7f32dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344fc26def8d36c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e1d9b700 1 -- 172.21.15.144:0/3554553183 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00e070 con 0x7f32e40cbdf0 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e1d9b700 1 -- 172.21.15.144:0/3554553183 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc00b9c0 con 0x7f32e40cbdf0 2023-11-28T11:51:23.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f32e1d9b700 1 -- 172.21.15.144:0/3554553183 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00bd50 con 0x7f32e40cbdf0 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a40052f0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7fe0c95df700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe0a80683c0 0x7fe0a806a870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40ccfc0 tx=0x7fe0c0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f19f0adf700 1 Processor -- start 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.020+0000 7f19f0adf700 1 -- start start 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19f0adf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec169890 0x7f19ec0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19f0adf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19ec0b51e0 0x7f19ec0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19f0adf700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0b6520 con 0x7f19ec0722a0 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19f0adf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec06c8a0 con 0x7f19ec0b51e0 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19f0adf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec09c4e0 con 0x7f19ec169890 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19ead9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19ec0b51e0 0x7f19ec0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19ead9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19ec0b51e0 0x7f19ec0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:36958/0 (socket says 172.21.15.144:36958) 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19ead9d700 1 -- 172.21.15.144:0/4191894271 learned_addr learned my addr 172.21.15.144:0/4191894271 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.021+0000 7f19e9d9b700 1 --1- 172.21.15.144:0/4191894271 >> v1:172.21.15.177:6789/0 conn(0x7f19ec0722a0 0x7f19ec0b4ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51884/0 (socket says 172.21.15.144:51884) 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19ead9d700 1 -- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec169890 msgr2=0x7f19ec0b7970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19ead9d700 1 --2- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec169890 0x7f19ec0b7970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19ead9d700 1 -- 172.21.15.144:0/4191894271 >> v1:172.21.15.177:6789/0 conn(0x7f19ec0722a0 legacy=0x7f19ec0b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19ead9d700 1 -- 172.21.15.144:0/4191894271 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0b5660 con 0x7f19ec0b51e0 2023-11-28T11:51:23.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19ead9d700 1 --2- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0b51e0 0x7f19ec0b05f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009e50 tx=0x7f19e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f97a5fe1bc74711 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19e959a700 1 -- 172.21.15.144:0/4191894271 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40092d0 con 0x7f19ec0b51e0 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19e959a700 1 -- 172.21.15.144:0/4191894271 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4030070 con 0x7f19ec0b51e0 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.022+0000 7f19e959a700 1 -- 172.21.15.144:0/4191894271 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400bc00 con 0x7f19ec0b51e0 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f697dffb700 1 -- 172.21.15.144:0/124663263 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974004230 con 0x7f6980164e60 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f697dffb700 1 -- 172.21.15.144:0/124663263 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974004390 con 0x7f6980164e60 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f697dffb700 1 -- 172.21.15.144:0/124663263 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740036a0 con 0x7f6980164e60 2023-11-28T11:51:23.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c0ce8f0 con 0x7fe77c0b0670 2023-11-28T11:51:23.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f19f0adf700 1 -- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0b51e0 msgr2=0x7f19ec0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f19f0adf700 1 --2- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0b51e0 0x7f19ec0b05f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009e50 tx=0x7f19e400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:23.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f6a91a30700 1 --2- 172.21.15.144:0/311685289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161b60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a880098f0 tx=0x7f6a88009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59aa1b41f72009a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f32e9069700 1 -- 172.21.15.144:0/3554553183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 msgr2=0x7f32e40ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f32e9069700 1 --2- 172.21.15.144:0/3554553183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 0x7f32e40ca650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009e50 tx=0x7f32dc00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:23.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f19f0adf700 1 -- 172.21.15.144:0/4191894271 shutdown_connections 2023-11-28T11:51:23.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f19f0adf700 1 --2- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0b51e0 0x7f19ec0b05f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f19f0adf700 1 --2- 172.21.15.144:0/4191894271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec169890 0x7f19ec0b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f19f0adf700 1 -- 172.21.15.144:0/4191894271 >> 172.21.15.144:0/4191894271 conn(0x7f19ec0cfad0 msgr2=0x7f19ec0cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f32e9069700 1 -- 172.21.15.144:0/3554553183 shutdown_connections 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f32e9069700 1 --2- 172.21.15.144:0/3554553183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e40cbdf0 0x7f32e40ca650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f32e9069700 1 --2- 172.21.15.144:0/3554553183 >> v2:172.21.15.177:3300/0 conn(0x7f32e406ba90 0x7f32e40af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.023+0000 7f32e9069700 1 -- 172.21.15.144:0/3554553183 >> 172.21.15.144:0/3554553183 conn(0x7f32e40d0000 msgr2=0x7f32e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f32e9069700 1 -- 172.21.15.144:0/3554553183 shutdown_connections 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f32e9069700 1 -- 172.21.15.144:0/3554553183 wait complete. 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f698567e700 1 -- 172.21.15.144:0/124663263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6980164e60 msgr2=0x7f69801652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f698567e700 1 --2- 172.21.15.144:0/124663263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6980164e60 0x7f69801652e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f69740098f0 tx=0x7f6974009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:23.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f19f0adf700 1 -- 172.21.15.144:0/4191894271 shutdown_connections 2023-11-28T11:51:23.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f19f0adf700 1 -- 172.21.15.144:0/4191894271 wait complete. 2023-11-28T11:51:23.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe770028020 con 0x7fe77c0b0670 2023-11-28T11:51:23.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7fe7637fe700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe764068460 0x7fe76406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe770087700 con 0x7fe77c0b0670 2023-11-28T11:51:23.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7fe77ad9d700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe764068460 0x7fe76406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f698567e700 1 -- 172.21.15.144:0/124663263 shutdown_connections 2023-11-28T11:51:23.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f698567e700 1 --2- 172.21.15.144:0/124663263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6980164e60 0x7f69801652e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.024+0000 7f698567e700 1 --2- 172.21.15.144:0/124663263 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69800ce480 0x7f69800c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.025+0000 7f698567e700 1 -- 172.21.15.144:0/124663263 >> 172.21.15.144:0/124663263 conn(0x7f69800cb670 msgr2=0x7f69800cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.025+0000 7fe77ad9d700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe764068460 0x7fe76406a910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe77c071b10 tx=0x7fe76c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.025+0000 7f698567e700 1 -- 172.21.15.144:0/124663263 shutdown_connections 2023-11-28T11:51:23.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.025+0000 7f698567e700 1 -- 172.21.15.144:0/124663263 wait complete. 2023-11-28T11:51:23.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 Processor -- start 2023-11-28T11:51:23.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f19f0adf700 1 Processor -- start 2023-11-28T11:51:23.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 -- start start 2023-11-28T11:51:23.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f19f0adf700 1 -- start start 2023-11-28T11:51:23.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32e406ba90 0x7f32e41ed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e40cbdf0 0x7f32e40b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 0x7f32e414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e415c690 con 0x7f32e40cbdf0 2023-11-28T11:51:23.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b49e0 con 0x7f32e406ba90 2023-11-28T11:51:23.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.026+0000 7f32e9069700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b4e10 con 0x7f32e415cef0 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19f0adf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0722a0 0x7f19ec0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19f0adf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec0b51e0 0x7f19ec14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19f0adf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 0x7f19ec0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19f0adf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b3030 con 0x7f19ec169890 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19f0adf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b3e90 con 0x7f19ec0722a0 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19f0adf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b0960 con 0x7f19ec0b51e0 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 0x7f19ec0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 0x7f19ec0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34706/0 (socket says 172.21.15.144:34706) 2023-11-28T11:51:23.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 -- 172.21.15.144:0/1473747838 learned_addr learned my addr 172.21.15.144:0/1473747838 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f698567e700 1 Processor -- start 2023-11-28T11:51:23.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f698567e700 1 -- start start 2023-11-28T11:51:23.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 -- 172.21.15.144:0/1473747838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec0b51e0 msgr2=0x7f19ec14db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec0b51e0 0x7f19ec14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 -- 172.21.15.144:0/1473747838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0722a0 msgr2=0x7f19ec0c6af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:23.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0722a0 0x7f19ec0c6af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.027+0000 7f19ead9d700 1 -- 172.21.15.144:0/1473747838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4009b30 con 0x7f19ec169890 2023-11-28T11:51:23.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f19ead9d700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 0x7f19ec0b4320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19e400b730 tx=0x7f19e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f698567e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 0x7f6980150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f698567e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6980161220 0x7f6980163700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f698567e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980164e60 0x7f6980163c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f698567e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69800b26a0 con 0x7f69800ce480 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f698567e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698014c3a0 con 0x7f6980164e60 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f698567e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69800ae940 con 0x7f6980161220 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 0x7f6980150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 0x7f6980150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34708/0 (socket says 172.21.15.144:34708) 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 -- 172.21.15.144:0/2538604807 learned_addr learned my addr 172.21.15.144:0/2538604807 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697e7fc700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6980161220 0x7f6980163700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f6a83fff700 1 -- 172.21.15.144:0/311685289 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88004230 con 0x7f6a8c0c2780 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f6a83fff700 1 -- 172.21.15.144:0/311685289 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a88004390 con 0x7f6a8c0c2780 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 -- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6980161220 msgr2=0x7f6980163700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6980161220 0x7f6980163700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 -- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980164e60 msgr2=0x7f6980163c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980164e60 0x7f6980163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f697effd700 1 -- 172.21.15.144:0/2538604807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009550 con 0x7f69800ce480 2023-11-28T11:51:23.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 0x7f32e414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 0x7f32e414f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46778/0 (socket says 172.21.15.144:46778) 2023-11-28T11:51:23.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 -- 172.21.15.144:0/3363955764 learned_addr learned my addr 172.21.15.144:0/3363955764 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f697effd700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 0x7f6980150b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f697800ba50 tx=0x7f697800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 -- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32e406ba90 msgr2=0x7f32e41ed080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:23.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32e406ba90 0x7f32e41ed080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 -- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e40cbdf0 msgr2=0x7f32e40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e40cbdf0 0x7f32e40b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 -- 172.21.15.144:0/3363955764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc009b30 con 0x7f32e415cef0 2023-11-28T11:51:23.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e359e700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 0x7f32e414f490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00b730 tx=0x7f32dc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0bc0521c0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0092e0 con 0x7f32e415cef0 2023-11-28T11:51:23.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc004950 con 0x7f32e415cef0 2023-11-28T11:51:23.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e415dff0 con 0x7f32e415cef0 2023-11-28T11:51:23.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e415dd10 con 0x7f32e415cef0 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.028+0000 7f6a83fff700 1 -- 172.21.15.144:0/311685289 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880036a0 con 0x7f6a8c0c2780 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f6a93493700 1 -- 172.21.15.144:0/311685289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 msgr2=0x7f6a8c161b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.029+0000 7f6a93493700 1 --2- 172.21.15.144:0/311685289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161b60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a880098f0 tx=0x7f6a88009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.030+0000 7f6a93493700 1 -- 172.21.15.144:0/311685289 shutdown_connections 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.030+0000 7f6a93493700 1 --2- 172.21.15.144:0/311685289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161b60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.030+0000 7f6a93493700 1 --2- 172.21.15.144:0/311685289 >> v2:172.21.15.177:3300/0 conn(0x7f6a8c0c2d40 0x7f6a8c161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.030+0000 7f6a93493700 1 -- 172.21.15.144:0/311685289 >> 172.21.15.144:0/311685289 conn(0x7f6a8c0b12b0 msgr2=0x7f6a8c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.030+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e40cab90 con 0x7f32e415cef0 2023-11-28T11:51:23.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.030+0000 7f6a93493700 1 -- 172.21.15.144:0/311685289 shutdown_connections 2023-11-28T11:51:23.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.031+0000 7f6a93493700 1 -- 172.21.15.144:0/311685289 wait complete. 2023-11-28T11:51:23.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.031+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc01c660 con 0x7f32e415cef0 2023-11-28T11:51:23.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.032+0000 7f6a93493700 1 Processor -- start 2023-11-28T11:51:23.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004250 con 0x7f19ec169890 2023-11-28T11:51:23.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e40043b0 con 0x7f19ec169890 2023-11-28T11:51:23.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4004760 con 0x7f19ec169890 2023-11-28T11:51:23.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0b58e0 con 0x7f19ec169890 2023-11-28T11:51:23.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec0b6040 con 0x7f19ec169890 2023-11-28T11:51:23.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe770053f20 con 0x7fe77c0b0670 2023-11-28T11:51:23.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.032+0000 7f6a93493700 1 -- start start 2023-11-28T11:51:23.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f6a93493700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f6a93493700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 0x7f6a8c0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f6a93493700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a8c164d10 0x7f6a8c0c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f6a93493700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1508b0 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f6a93493700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0c94e0 con 0x7f6a8c164d10 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f6a93493700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0b31a0 con 0x7f6a8c0c2780 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 0x7f6a8c0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 0x7f6a8c0c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34718/0 (socket says 172.21.15.144:34718) 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 -- 172.21.15.144:0/1422157792 learned_addr learned my addr 172.21.15.144:0/1422157792 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.033+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32dc022030 con 0x7f32e415cef0 2023-11-28T11:51:23.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f32cb7fe700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f32d00682f0 0x7f32d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32dc087a70 con 0x7f32e415cef0 2023-11-28T11:51:23.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f32e2d9d700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f32d00682f0 0x7f32d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.035+0000 7f32e2d9d700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f32d00682f0 0x7f32d006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f32d8000f60 tx=0x7f32d8007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.035+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19e404e070 con 0x7f19ec169890 2023-11-28T11:51:23.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 -- 172.21.15.144:0/1422157792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 msgr2=0x7f6a8c161890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:23.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 -- 172.21.15.144:0/1422157792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a8c164d10 msgr2=0x7f6a8c0c37b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:23.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.034+0000 7f6a90a2e700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a8c164d10 0x7f6a8c0c37b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.035+0000 7f6a90a2e700 1 -- 172.21.15.144:0/1422157792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a88009550 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.035+0000 7f6a90a2e700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 0x7f6a8c0c8b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007ef0 tx=0x7f6a8400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800c9e0 con 0x7f69800ce480 2023-11-28T11:51:23.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697800cb40 con 0x7f69800ce480 2023-11-28T11:51:23.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978014440 con 0x7f69800ce480 2023-11-28T11:51:23.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6980161f30 con 0x7f69800ce480 2023-11-28T11:51:23.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6980161c50 con 0x7f69800ce480 2023-11-28T11:51:23.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f19daffd700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19cc0682f0 0x7f19cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.036+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19e40a95a0 con 0x7f19ec169890 2023-11-28T11:51:23.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.037+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32dc0541e0 con 0x7f32e415cef0 2023-11-28T11:51:23.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.037+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69600052f0 con 0x7f69800ce480 2023-11-28T11:51:23.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.037+0000 7f19ea59c700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19cc0682f0 0x7f19cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.039+0000 7f19ea59c700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19cc0682f0 0x7f19cc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009710 tx=0x7f19dc006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.039+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f697801b030 con 0x7f69800ce480 2023-11-28T11:51:23.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.039+0000 7f69677fe700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f696c068340 0x7f696c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.039+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6978085b30 con 0x7f69800ce480 2023-11-28T11:51:23.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.040+0000 7f697e7fc700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f696c068340 0x7f696c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.040+0000 7f697e7fc700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f696c068340 0x7f696c06a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6970007fd0 tx=0x7f6970009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.043+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f69780524e0 con 0x7f69800ce480 2023-11-28T11:51:23.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.048+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d00052f0 con 0x7f19ec169890 2023-11-28T11:51:23.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.049+0000 7fedd1564700 1 Processor -- start 2023-11-28T11:51:23.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.049+0000 7fedd1564700 1 -- start start 2023-11-28T11:51:23.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.050+0000 7fedd1564700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc158890 0x7fedcc0ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.050+0000 7fedd1564700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc15c490 con 0x7fedcc0b0d80 2023-11-28T11:51:23.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.050+0000 7fedd1564700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0ce650 con 0x7fedcc06d4f0 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.050+0000 7fedd1564700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15c730 con 0x7fedcc158890 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.050+0000 7fedcb7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fedcc0b0d80 0x7fedcc06b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51442/0 (socket says 172.21.15.144:51442) 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.050+0000 7fedcb7fe700 1 -- 172.21.15.144:0/1783808023 learned_addr learned my addr 172.21.15.144:0/1783808023 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedcaffd700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc158890 0x7fedcc0ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051129045 0 0) 0x7fedcc0ce650 con 0x7fedcc06d4f0 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb40035b0 con 0x7fedcc06d4f0 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802408387 0 0) 0x7fedb40035b0 con 0x7fedcc06d4f0 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc158890 msgr2=0x7fedcc0ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc158890 0x7fedcc0ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 >> v1:172.21.15.144:6789/0 conn(0x7fedcc0b0d80 legacy=0x7fedcc06b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc15ca10 con 0x7fedcc06d4f0 2023-11-28T11:51:23.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fedc4001a30 con 0x7fedcc06d4f0 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 >> v1:172.21.15.177:6789/0 conn(0x7fedcc06d4f0 legacy=0x7fedcc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.051+0000 7fedc9ffb700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedb4003a00 0x7fedb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedc9ffb700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb4004850 0x7fedb4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedc9ffb700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb4005200 0x7fedb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15c730 con 0x7fedb4003a00 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb400c030 con 0x7fedb4004850 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb400c1f0 con 0x7fedb4005200 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedca7fc700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb4004850 0x7fedb4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedca7fc700 1 -- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb4005200 msgr2=0x7fedb400ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedca7fc700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedb4005200 0x7fedb400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedca7fc700 1 -- 172.21.15.144:0/1783808023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedb4003a00 msgr2=0x7fedb4003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:51:23.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedca7fc700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedb4003a00 0x7fedb4003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.052+0000 7fedca7fc700 1 -- 172.21.15.144:0/1783808023 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc15ca10 con 0x7fedb4004850 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.053+0000 7fedca7fc700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb4004850 0x7fedb4004cc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fedcc06d8c0 tx=0x7fedb4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1813c9e216075bad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.053+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4019600 con 0x7fedb4004850 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.053+0000 7fedc9ffb700 1 -- 172.21.15.144:0/1783808023 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4019760 con 0x7fedb4004850 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.054+0000 7fedd1564700 1 -- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb4004850 msgr2=0x7fedb4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.054+0000 7fedd1564700 1 --2- 172.21.15.144:0/1783808023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb4004850 0x7fedb4004cc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fedcc06d8c0 tx=0x7fedb4005880 comp rx=0 tx=0).stop 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.054+0000 7fedcb7fe700 1 -- 172.21.15.144:0/1783808023 reap_dead start 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.054+0000 7fedd1564700 1 -- 172.21.15.144:0/1783808023 shutdown_connections 2023-11-28T11:51:23.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.054+0000 7fedd1564700 1 -- 172.21.15.144:0/1783808023 >> 172.21.15.144:0/1783808023 conn(0x7fedcc0cb670 msgr2=0x7fedcc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.057+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19e4075d10 con 0x7f19ec169890 2023-11-28T11:51:23.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.058+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84014420 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.058+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a84014580 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.058+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84014910 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.058+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c0c9eb0 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.058+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c0cc000 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.059+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c0ca190 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.062+0000 7fedd1564700 1 -- 172.21.15.144:0/1783808023 shutdown_connections 2023-11-28T11:51:23.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.062+0000 7fedd1564700 1 -- 172.21.15.144:0/1783808023 wait complete. 2023-11-28T11:51:23.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.064+0000 7fedd1564700 1 Processor -- start 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.064+0000 7fedd1564700 1 -- start start 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.065+0000 7fedd1564700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc154250 0x7fedcc1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.065+0000 7fedd1564700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc15b3e0 0x7fedcc1e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.065+0000 7fedd1564700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 0x7fedcc1e2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.065+0000 7fedd1564700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1f78e0 con 0x7fedcc154250 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.065+0000 7fedd1564700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1e28c0 con 0x7fedcc1e1f00 2023-11-28T11:51:23.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.065+0000 7fedd1564700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1e2d80 con 0x7fedcc15b3e0 2023-11-28T11:51:23.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.073+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a84012050 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.073+0000 7f6a81ffb700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a70068340 0x7f6a7006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.073+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a840872c0 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.073+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a84087680 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.074+0000 7f6a9122f700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a70068340 0x7f6a7006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 0x7fedcc1e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 0x7fedcc1e2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:37012/0 (socket says 172.21.15.144:37012) 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 -- 172.21.15.144:0/3048070246 learned_addr learned my addr 172.21.15.144:0/3048070246 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 -- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc15b3e0 msgr2=0x7fedcc1e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc15b3e0 0x7fedcc1e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 -- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc154250 msgr2=0x7fedcc1f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc154250 0x7fedcc1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 -- 172.21.15.144:0/3048070246 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0c9370 con 0x7fedcc1e1f00 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.066+0000 7fedcb7fe700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 0x7fedcc1e2380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0c9df0 tx=0x7fedbc00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.075+0000 7f6a9122f700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a70068340 0x7f6a7006a7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c001590 tx=0x7f6a7c007a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.075+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00f810 con 0x7fedcc1e1f00 2023-11-28T11:51:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.075+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc00f970 con 0x7fedcc1e1f00 2023-11-28T11:51:23.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.075+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00fce0 con 0x7fedcc1e1f00 2023-11-28T11:51:23.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.075+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc15ccf0 con 0x7fedcc1e1f00 2023-11-28T11:51:23.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.075+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc158c60 con 0x7fedcc1e1f00 2023-11-28T11:51:23.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.076+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0052f0 con 0x7fedcc1e1f00 2023-11-28T11:51:23.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.082+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedbc01f070 con 0x7fedcc1e1f00 2023-11-28T11:51:23.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.082+0000 7fedb37fe700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedb80683c0 0x7fedb806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:23.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.082+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fedbc085ed0 con 0x7fedcc1e1f00 2023-11-28T11:51:23.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.083+0000 7fedcaffd700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedb80683c0 0x7fedb806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:23.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.084+0000 7fedcaffd700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedb80683c0 0x7fedb806a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0c8520 tx=0x7fedc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:23.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.084+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedbc052660 con 0x7fedcc1e1f00 2023-11-28T11:51:23.406 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.406+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f32e40d0e60 con 0x7f32e415cef0 2023-11-28T11:51:23.407 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.406+0000 7f32cb7fe700 1 -- 172.21.15.144:0/3363955764 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32dc087070 con 0x7f32e415cef0 2023-11-28T11:51:23.410 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.410 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.409+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6960005870 con 0x7f69800ce480 2023-11-28T11:51:23.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f32d00682f0 msgr2=0x7f32d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f32d00682f0 0x7f32d006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f32d8000f60 tx=0x7f32d8007a40 comp rx=0 tx=0).stop 2023-11-28T11:51:23.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 msgr2=0x7f32e414f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 0x7f32e414f490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00b730 tx=0x7f32dc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f69677fe700 1 -- 172.21.15.144:0/2538604807 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6978019020 con 0x7f69800ce480 2023-11-28T11:51:23.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 shutdown_connections 2023-11-28T11:51:23.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f32d00682f0 0x7f32d006a7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32e415cef0 0x7f32e414f490 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32e406ba90 0x7f32e41ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 --2- 172.21.15.144:0/3363955764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e40cbdf0 0x7f32e40b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.410+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 >> 172.21.15.144:0/3363955764 conn(0x7f32e40d0000 msgr2=0x7f32e406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.412 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.413+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 shutdown_connections 2023-11-28T11:51:23.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.413+0000 7f32e9069700 1 -- 172.21.15.144:0/3363955764 wait complete. 2023-11-28T11:51:23.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.414+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f696c068340 msgr2=0x7f696c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.414+0000 7f698567e700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f696c068340 0x7f696c06a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6970007fd0 tx=0x7f6970009040 comp rx=0 tx=0).stop 2023-11-28T11:51:23.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.414+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 msgr2=0x7f6980150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.414+0000 7f698567e700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 0x7f6980150b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f697800ba50 tx=0x7f697800bd60 comp rx=0 tx=0).stop 2023-11-28T11:51:23.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 shutdown_connections 2023-11-28T11:51:23.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7f698567e700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69800ce480 0x7f6980150b20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7f698567e700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f696c068340 0x7f696c06a7f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7f698567e700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980164e60 0x7f6980163c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7f698567e700 1 --2- 172.21.15.144:0/2538604807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6980161220 0x7f6980163700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 >> 172.21.15.144:0/2538604807 conn(0x7f69800cb670 msgr2=0x7f698015ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd63c1457e0 con 0x7fd63c160b30 2023-11-28T11:51:23.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.416+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe0a40055a0 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.417 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.417+0000 7fe0b9ffb700 1 -- 172.21.15.144:0/3995735075 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0bc019070 con 0x7fe0c40b5ea0 2023-11-28T11:51:23.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.417+0000 7fd631ffb700 1 -- 172.21.15.144:0/356907878 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd63c1457e0 con 0x7fd63c160b30 2023-11-28T11:51:23.419 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.419+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe77c0ce8f0 con 0x7fe77c0b0670 2023-11-28T11:51:23.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.420+0000 7fe7637fe700 1 -- 172.21.15.144:0/1221131408 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe770057540 con 0x7fe77c0b0670 2023-11-28T11:51:23.420 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200683c0 msgr2=0x7fd62006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200683c0 0x7fd62006a870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd62c007fd0 tx=0x7fd62c009040 comp rx=0 tx=0).stop 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 msgr2=0x7fd63c1411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 0x7fd63c1411b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c480 tx=0x7fd63800c790 comp rx=0 tx=0).stop 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 shutdown_connections 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200683c0 0x7fd62006a870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c160f00 0x7fd63c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0b0d80 0x7fd63c140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 --2- 172.21.15.144:0/356907878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c160b30 0x7fd63c1411b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 >> 172.21.15.144:0/356907878 conn(0x7fd63c0cb670 msgr2=0x7fd63c0ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 shutdown_connections 2023-11-28T11:51:23.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fd6439fe700 1 -- 172.21.15.144:0/356907878 wait complete. 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.422+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe764068460 msgr2=0x7fe76406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.422+0000 7fe781111700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe764068460 0x7fe76406a910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe77c071b10 tx=0x7fe76c009040 comp rx=0 tx=0).stop 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.422+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 msgr2=0x7fe77c154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.422+0000 7fe781111700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 0x7fe77c154e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe770009fd0 tx=0x7fe7700047c0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 shutdown_connections 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fe781111700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe764068460 0x7fe76406a910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fe781111700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe77c0cc180 0x7fe77c070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fe781111700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe77c06d4f0 0x7fe77c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fe781111700 1 --2- 172.21.15.144:0/1221131408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c0b0670 0x7fe77c154e20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.423+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 >> 172.21.15.144:0/1221131408 conn(0x7fe77c0cb670 msgr2=0x7fe77c157140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.424+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 shutdown_connections 2023-11-28T11:51:23.427 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.424+0000 7fe781111700 1 -- 172.21.15.144:0/1221131408 wait complete. 2023-11-28T11:51:23.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.426+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 shutdown_connections 2023-11-28T11:51:23.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.426+0000 7f698567e700 1 -- 172.21.15.144:0/2538604807 wait complete. 2023-11-28T11:51:23.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2023-11-28T11:51:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.437+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f19d00055a0 con 0x7f19ec169890 2023-11-28T11:51:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.438+0000 7f19daffd700 1 -- 172.21.15.144:0/1473747838 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f19e4079330 con 0x7f19ec169890 2023-11-28T11:51:23.439 INFO:teuthology.orchestra.run.smithi144.stdout:60129542148 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.442+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19cc0682f0 msgr2=0x7f19cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.442+0000 7f19f0adf700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19cc0682f0 0x7f19cc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009710 tx=0x7f19dc006c60 comp rx=0 tx=0).stop 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.442+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 msgr2=0x7f19ec0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.442+0000 7f19f0adf700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 0x7f19ec0b4320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f19e400b730 tx=0x7f19e4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 shutdown_connections 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ec169890 0x7f19ec0b4320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19cc0682f0 0x7f19cc06a7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec0722a0 0x7f19ec0c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 --2- 172.21.15.144:0/1473747838 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19ec0b51e0 0x7f19ec14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 >> 172.21.15.144:0/1473747838 conn(0x7f19ec0cfad0 msgr2=0x7f19ec0d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 shutdown_connections 2023-11-28T11:51:23.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.443+0000 7f19f0adf700 1 -- 172.21.15.144:0/1473747838 wait complete. 2023-11-28T11:51:23.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.437+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe0a80683c0 msgr2=0x7fe0a806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.437+0000 7fe0cb843700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe0a80683c0 0x7fe0a806a870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40ccfc0 tx=0x7fe0c0009190 comp rx=0 tx=0).stop 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.437+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b5ea0 msgr2=0x7fe0c40c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.437+0000 7fe0cb843700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b5ea0 0x7fe0c40c8630 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00bdf0 tx=0x7fe0bc009f70 comp rx=0 tx=0).stop 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 shutdown_connections 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0c40b51e0 0x7fe0c40c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe0a80683c0 0x7fe0a806a870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0c40c96b0 0x7fe0c41fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 --2- 172.21.15.144:0/3995735075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0c40b5ea0 0x7fe0c40c8630 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 >> 172.21.15.144:0/3995735075 conn(0x7fe0c40cfad0 msgr2=0x7fe0c40b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.444+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 shutdown_connections 2023-11-28T11:51:23.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.445+0000 7fe0cb843700 1 -- 172.21.15.144:0/3995735075 wait complete. 2023-11-28T11:51:23.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2023-11-28T11:51:23.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2023-11-28T11:51:23.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.445+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6a8c066110 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.446+0000 7f6a81ffb700 1 -- 172.21.15.144:0/1422157792 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a84053ab0 con 0x7f6a8c0c2d40 2023-11-28T11:51:23.449 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a70068340 msgr2=0x7f6a7006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a70068340 0x7f6a7006a7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c001590 tx=0x7f6a7c007a20 comp rx=0 tx=0).stop 2023-11-28T11:51:23.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 msgr2=0x7f6a8c0c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 0x7f6a8c0c8b00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007ef0 tx=0x7f6a8400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 shutdown_connections 2023-11-28T11:51:23.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a70068340 0x7f6a7006a7f0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c001590 tx=0x7f6a7c007a20 comp rx=0 tx=0).stop 2023-11-28T11:51:23.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a8c164d10 0x7f6a8c0c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a8c0c2780 0x7f6a8c161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 --2- 172.21.15.144:0/1422157792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8c0c2d40 0x7f6a8c0c8b00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.448+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 >> 172.21.15.144:0/1422157792 conn(0x7f6a8c0b12b0 msgr2=0x7f6a8c06a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.1 2023-11-28T11:51:23.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.455+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 shutdown_connections 2023-11-28T11:51:23.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.455+0000 7f6a93493700 1 -- 172.21.15.144:0/1422157792 wait complete. 2023-11-28T11:51:23.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.460+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fedac0055a0 con 0x7fedcc1e1f00 2023-11-28T11:51:23.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2023-11-28T11:51:23.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.460+0000 7fedb37fe700 1 -- 172.21.15.144:0/3048070246 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fedbc01d030 con 0x7fedcc1e1f00 2023-11-28T11:51:23.462 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:23.464 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedb80683c0 msgr2=0x7fedb806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.464 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedb80683c0 0x7fedb806a870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0c8520 tx=0x7fedc0008040 comp rx=0 tx=0).stop 2023-11-28T11:51:23.464 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 msgr2=0x7fedcc1e2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 0x7fedcc1e2380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0c9df0 tx=0x7fedbc00b8e0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 shutdown_connections 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc154250 0x7fedcc1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedb80683c0 0x7fedb806a870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedcc1e1f00 0x7fedcc1e2380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 --2- 172.21.15.144:0/3048070246 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedcc15b3e0 0x7fedcc1e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 >> 172.21.15.144:0/3048070246 conn(0x7fedcc0cb670 msgr2=0x7fedcc070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 shutdown_connections 2023-11-28T11:51:23.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:23.463+0000 7fedd1564700 1 -- 172.21.15.144:0/3048070246 wait complete. 2023-11-28T11:51:23.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2023-11-28T11:51:23.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2023-11-28T11:51:23.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2023-11-28T11:51:24.439 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:51:24.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:51:24.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:51:24.453 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:51:24.462 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:51:24.467 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:51:24.472 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:51:24.478 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:51:24.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.647+0000 7fc161528700 1 Processor -- start 2023-11-28T11:51:24.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.648+0000 7fc161528700 1 -- start start 2023-11-28T11:51:24.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.649+0000 7fc161528700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c0b0750 0x7fc15c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.649+0000 7fc161528700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc15c06c660 0x7fc15c1646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.649+0000 7fc161528700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0719b0 con 0x7fc15c164c30 2023-11-28T11:51:24.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.649+0000 7fc161528700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c1656c0 con 0x7fc15c06c660 2023-11-28T11:51:24.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.649+0000 7fc161528700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c1653e0 con 0x7fc15c0b0750 2023-11-28T11:51:24.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.650+0000 7fc15b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc15c164c30 0x7fc15c06fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51460/0 (socket says 172.21.15.144:51460) 2023-11-28T11:51:24.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.650+0000 7fc15b7fe700 1 -- 172.21.15.144:0/216096955 learned_addr learned my addr 172.21.15.144:0/216096955 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:24.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.650+0000 7fc15a7fc700 1 --2- 172.21.15.144:0/216096955 >> v2:172.21.15.144:3300/0 conn(0x7fc15c06c660 0x7fc15c1646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.651+0000 7fc15a7fc700 1 -- 172.21.15.144:0/216096955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c0b0750 msgr2=0x7fc15c0b0b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:24.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.651+0000 7fc15a7fc700 1 --2- 172.21.15.144:0/216096955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c0b0750 0x7fc15c0b0b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.651+0000 7fc15a7fc700 1 -- 172.21.15.144:0/216096955 >> v1:172.21.15.144:6789/0 conn(0x7fc15c164c30 legacy=0x7fc15c06fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.651+0000 7fc15a7fc700 1 -- 172.21.15.144:0/216096955 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c06bb80 con 0x7fc15c06c660 2023-11-28T11:51:24.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.651+0000 7fc15a7fc700 1 --2- 172.21.15.144:0/216096955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c06c660 0x7fc15c1646f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009840 tx=0x7fc14c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f20b3f75b8d6647 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.652+0000 7fc159ffb700 1 -- 172.21.15.144:0/216096955 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c011830 con 0x7fc15c06c660 2023-11-28T11:51:24.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.652+0000 7fc159ffb700 1 -- 172.21.15.144:0/216096955 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c011990 con 0x7fc15c06c660 2023-11-28T11:51:24.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.652+0000 7fc159ffb700 1 -- 172.21.15.144:0/216096955 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c011d20 con 0x7fc15c06c660 2023-11-28T11:51:24.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.653+0000 7fc161528700 1 -- 172.21.15.144:0/216096955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c06c660 msgr2=0x7fc15c1646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.653+0000 7fc161528700 1 --2- 172.21.15.144:0/216096955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c06c660 0x7fc15c1646f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009840 tx=0x7fc14c009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:24.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.653+0000 7fc161528700 1 -- 172.21.15.144:0/216096955 shutdown_connections 2023-11-28T11:51:24.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.653+0000 7fc161528700 1 --2- 172.21.15.144:0/216096955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c0b0750 0x7fc15c0b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.653+0000 7fc161528700 1 --2- 172.21.15.144:0/216096955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c06c660 0x7fc15c1646f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.653+0000 7fc161528700 1 -- 172.21.15.144:0/216096955 >> 172.21.15.144:0/216096955 conn(0x7fc15c06aeb0 msgr2=0x7fc15c0cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.657+0000 7fc161528700 1 -- 172.21.15.144:0/216096955 shutdown_connections 2023-11-28T11:51:24.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.657+0000 7fc161528700 1 -- 172.21.15.144:0/216096955 wait complete. 2023-11-28T11:51:24.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.659+0000 7fc161528700 1 Processor -- start 2023-11-28T11:51:24.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.660+0000 7fc161528700 1 -- start start 2023-11-28T11:51:24.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.661+0000 7fc161528700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 0x7fc15c0cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.661+0000 7fc161528700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c0b0750 0x7fc15c0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.661+0000 7fc161528700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c164c30 0x7fc15c1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.661+0000 7fc161528700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0c5450 con 0x7fc15c0b0750 2023-11-28T11:51:24.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.661+0000 7fc161528700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c14f7a0 con 0x7fc15c06c660 2023-11-28T11:51:24.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.661+0000 7fc161528700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c14ec00 con 0x7fc15c164c30 2023-11-28T11:51:24.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.662+0000 7fc15affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 0x7fc15c0cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.662+0000 7fc15affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 0x7fc15c0cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:37030/0 (socket says 172.21.15.144:37030) 2023-11-28T11:51:24.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.662+0000 7fc15affd700 1 -- 172.21.15.144:0/3285894769 learned_addr learned my addr 172.21.15.144:0/3285894769 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.662+0000 7fc15b7fe700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c164c30 0x7fc15c1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc15affd700 1 -- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c164c30 msgr2=0x7fc15c1510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc15affd700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c164c30 0x7fc15c1510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc15affd700 1 -- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c0b0750 msgr2=0x7fc15c0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc15affd700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c0b0750 0x7fc15c0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc15affd700 1 -- 172.21.15.144:0/3285894769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c0094a0 con 0x7fc15c06c660 2023-11-28T11:51:24.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc15affd700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 0x7fc15c0cc880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc15000b580 tx=0x7fc15000b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.663+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150011830 con 0x7fc15c06c660 2023-11-28T11:51:24.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.664+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500119b0 con 0x7fc15c06c660 2023-11-28T11:51:24.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.664+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150011d60 con 0x7fc15c06c660 2023-11-28T11:51:24.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.664+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c06c140 con 0x7fc15c06c660 2023-11-28T11:51:24.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.664+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c06be60 con 0x7fc15c06c660 2023-11-28T11:51:24.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.665+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c0b1060 con 0x7fc15c06c660 2023-11-28T11:51:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.666+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc150021070 con 0x7fc15c06c660 2023-11-28T11:51:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.667+0000 7fc1437fe700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1480683c0 0x7fc14806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.667+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc150085b20 con 0x7fc15c06c660 2023-11-28T11:51:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.667+0000 7fc15a7fc700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1480683c0 0x7fc14806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.668+0000 7fc15a7fc700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1480683c0 0x7fc14806a870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0090b0 tx=0x7fc14c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.678+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc150052310 con 0x7fc15c06c660 2023-11-28T11:51:24.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.697+0000 7f6383854700 1 Processor -- start 2023-11-28T11:51:24.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.697+0000 7f6383854700 1 -- start start 2023-11-28T11:51:24.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6383854700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f637c0cacb0 0x7f637c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6383854700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cde10 0x7f637c0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6383854700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f637c0b0670 0x7f637c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6383854700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0ce7a0 con 0x7f637c0cacb0 2023-11-28T11:51:24.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6383854700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0b1030 con 0x7f637c0b0670 2023-11-28T11:51:24.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6383854700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c06c2f0 con 0x7f637c0cde10 2023-11-28T11:51:24.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f63815f0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f637c0cacb0 0x7f637c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f63815f0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f637c0cacb0 0x7f637c0cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34760/0 (socket says 172.21.15.144:34760) 2023-11-28T11:51:24.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f6380def700 1 --2- 172.21.15.144:0/3040842555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cde10 0x7f637c0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.698+0000 7f63815f0700 1 -- 172.21.15.144:0/3040842555 learned_addr learned my addr 172.21.15.144:0/3040842555 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.699+0000 7f63815f0700 1 -- 172.21.15.144:0/3040842555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cde10 msgr2=0x7f637c0ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.699+0000 7f63815f0700 1 --2- 172.21.15.144:0/3040842555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cde10 0x7f637c0ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.699+0000 7f63815f0700 1 -- 172.21.15.144:0/3040842555 >> v2:172.21.15.177:3300/0 conn(0x7f637c0b0670 msgr2=0x7f637c0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.699+0000 7f6381df1700 1 --2- 172.21.15.144:0/3040842555 >> v2:172.21.15.177:3300/0 conn(0x7f637c0b0670 0x7f637c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.700+0000 7f63815f0700 1 --2- 172.21.15.144:0/3040842555 >> v2:172.21.15.177:3300/0 conn(0x7f637c0b0670 0x7f637c0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.700+0000 7f63815f0700 1 -- 172.21.15.144:0/3040842555 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c06b9c0 con 0x7f637c0cacb0 2023-11-28T11:51:24.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.700+0000 7f6381df1700 1 --2- 172.21.15.144:0/3040842555 >> v2:172.21.15.177:3300/0 conn(0x7f637c0b0670 0x7f637c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:24.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.705+0000 7f63815f0700 1 --2- 172.21.15.144:0/3040842555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0cacb0 0x7f637c0cb080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f637c0ae5e0 tx=0x7f637000f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619ca8a061944a9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.705+0000 7f636ffff700 1 -- 172.21.15.144:0/3040842555 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370012070 con 0x7f637c0cacb0 2023-11-28T11:51:24.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.705+0000 7f636ffff700 1 -- 172.21.15.144:0/3040842555 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637000d2e0 con 0x7f637c0cacb0 2023-11-28T11:51:24.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.705+0000 7f636ffff700 1 -- 172.21.15.144:0/3040842555 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000f950 con 0x7f637c0cacb0 2023-11-28T11:51:24.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.706+0000 7fc700eab700 1 Processor -- start 2023-11-28T11:51:24.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.706+0000 7fc700eab700 1 -- start start 2023-11-28T11:51:24.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.706+0000 7f6383854700 1 -- 172.21.15.144:0/3040842555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0cacb0 msgr2=0x7f637c0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.706+0000 7f6383854700 1 --2- 172.21.15.144:0/3040842555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0cacb0 0x7f637c0cb080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f637c0ae5e0 tx=0x7f637000f050 comp rx=0 tx=0).stop 2023-11-28T11:51:24.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc700eab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6fc159860 0x7fc6fc0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc700eab700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc6fc0b0d80 0x7fc6fc0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc700eab700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc0ccc00 con 0x7fc6fc159490 2023-11-28T11:51:24.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc700eab700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0ccec0 con 0x7fc6fc159860 2023-11-28T11:51:24.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc700eab700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0cd1a0 con 0x7fc6fc0b0d80 2023-11-28T11:51:24.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc6f9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6fc159860 0x7fc6fc0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc6fa59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc6fc159490 0x7fc6fc0b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51478/0 (socket says 172.21.15.144:51478) 2023-11-28T11:51:24.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc6f9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6fc159860 0x7fc6fc0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34770/0 (socket says 172.21.15.144:34770) 2023-11-28T11:51:24.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.707+0000 7fc6fa59c700 1 -- 172.21.15.144:0/2118481804 learned_addr learned my addr 172.21.15.144:0/2118481804 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:24.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f959a700 1 -- 172.21.15.144:0/2118481804 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991307290 0 0) 0x7fc6fc0ccc00 con 0x7fc6fc159490 2023-11-28T11:51:24.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f959a700 1 -- 172.21.15.144:0/2118481804 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e40035b0 con 0x7fc6fc159490 2023-11-28T11:51:24.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f9d9b700 1 -- 172.21.15.144:0/2118481804 >> v2:172.21.15.177:3300/0 conn(0x7fc6fc0b0d80 msgr2=0x7fc6fc0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f9d9b700 1 --2- 172.21.15.144:0/2118481804 >> v2:172.21.15.177:3300/0 conn(0x7fc6fc0b0d80 0x7fc6fc0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f9d9b700 1 -- 172.21.15.144:0/2118481804 >> v1:172.21.15.144:6789/0 conn(0x7fc6fc159490 legacy=0x7fc6fc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f9d9b700 1 -- 172.21.15.144:0/2118481804 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0cd480 con 0x7fc6fc159860 2023-11-28T11:51:24.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7f6383854700 1 -- 172.21.15.144:0/3040842555 shutdown_connections 2023-11-28T11:51:24.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7f6383854700 1 --2- 172.21.15.144:0/3040842555 >> v2:172.21.15.177:3300/0 conn(0x7f637c0b0670 0x7f637c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7f6383854700 1 --2- 172.21.15.144:0/3040842555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0cacb0 0x7f637c0cb080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7f6383854700 1 --2- 172.21.15.144:0/3040842555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cde10 0x7f637c0ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7f6383854700 1 -- 172.21.15.144:0/3040842555 >> 172.21.15.144:0/3040842555 conn(0x7f637c0cb670 msgr2=0x7f637c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f9d9b700 1 --2- 172.21.15.144:0/2118481804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc159860 0x7fc6fc0c94c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0c5be0 tx=0x7fc6ec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bb165e70882f4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f959a700 1 -- 172.21.15.144:0/2118481804 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec011830 con 0x7fc6fc159860 2023-11-28T11:51:24.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f959a700 1 -- 172.21.15.144:0/2118481804 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec011990 con 0x7fc6fc159860 2023-11-28T11:51:24.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.708+0000 7fc6f959a700 1 -- 172.21.15.144:0/2118481804 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec011d20 con 0x7fc6fc159860 2023-11-28T11:51:24.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.709+0000 7ff355998700 1 Processor -- start 2023-11-28T11:51:24.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.709+0000 7fc700eab700 1 -- 172.21.15.144:0/2118481804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc159860 msgr2=0x7fc6fc0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.709+0000 7fc700eab700 1 --2- 172.21.15.144:0/2118481804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc159860 0x7fc6fc0c94c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0c5be0 tx=0x7fc6ec009b20 comp rx=0 tx=0).stop 2023-11-28T11:51:24.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.709+0000 7ff355998700 1 -- start start 2023-11-28T11:51:24.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.709+0000 7ff355998700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff35006c990 0x7ff35006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.709+0000 7ff355998700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3500ca240 0x7ff3500ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff355998700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35006ae60 con 0x7ff35006e9f0 2023-11-28T11:51:24.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff355998700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35006d350 con 0x7ff35006c990 2023-11-28T11:51:24.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff355998700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cabc0 con 0x7ff3500ca240 2023-11-28T11:51:24.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34e7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff35006c990 0x7ff35006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34effd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff35006e9f0 0x7ff35006b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:51974/0 (socket says 172.21.15.144:51974) 2023-11-28T11:51:24.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34effd700 1 -- 172.21.15.144:0/495795531 learned_addr learned my addr 172.21.15.144:0/495795531 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:24.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34f7fe700 1 --2- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3500ca240 0x7ff3500ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7fc700eab700 1 -- 172.21.15.144:0/2118481804 shutdown_connections 2023-11-28T11:51:24.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7fc700eab700 1 --2- 172.21.15.144:0/2118481804 >> v2:172.21.15.177:3300/0 conn(0x7fc6fc0b0d80 0x7fc6fc0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7fc700eab700 1 --2- 172.21.15.144:0/2118481804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc159860 0x7fc6fc0c94c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7fc700eab700 1 -- 172.21.15.144:0/2118481804 >> 172.21.15.144:0/2118481804 conn(0x7fc6fc0cb670 msgr2=0x7fc6fc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7f1262323700 1 Processor -- start 2023-11-28T11:51:24.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7f1262323700 1 -- start start 2023-11-28T11:51:24.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34e7fc700 1 -- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3500ca240 msgr2=0x7ff3500ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34e7fc700 1 --2- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3500ca240 0x7ff3500ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34e7fc700 1 -- 172.21.15.144:0/495795531 >> v1:172.21.15.177:6789/0 conn(0x7ff35006e9f0 legacy=0x7ff35006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34e7fc700 1 -- 172.21.15.144:0/495795531 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35006a270 con 0x7ff35006c990 2023-11-28T11:51:24.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7fc700eab700 1 -- 172.21.15.144:0/2118481804 shutdown_connections 2023-11-28T11:51:24.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7fc700eab700 1 -- 172.21.15.144:0/2118481804 wait complete. 2023-11-28T11:51:24.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.710+0000 7ff34e7fc700 1 --2- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff35006c990 0x7ff35006ce10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff340009840 tx=0x7ff340009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b8c9d6c8b34b6d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.711+0000 7f1262323700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160b30 0x7f125c0c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.711+0000 7f1262323700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f125c160f00 0x7f125c164840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.711+0000 7f1262323700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f125c0cc730 0x7f125c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.711+0000 7f1262323700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125c164d80 con 0x7f125c160f00 2023-11-28T11:51:24.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.711+0000 7f1262323700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125c06ba50 con 0x7f125c0cc730 2023-11-28T11:51:24.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.711+0000 7f1262323700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125c164f90 con 0x7f125c160b30 2023-11-28T11:51:24.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f125c160f00 0x7f125c164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f125c160f00 0x7f125c164840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34782/0 (socket says 172.21.15.144:34782) 2023-11-28T11:51:24.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 -- 172.21.15.144:0/808223812 learned_addr learned my addr 172.21.15.144:0/808223812 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 -- 172.21.15.144:0/808223812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160b30 msgr2=0x7f125c0c1e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:24.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 --2- 172.21.15.144:0/808223812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160b30 0x7f125c0c1e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 -- 172.21.15.144:0/808223812 >> v2:172.21.15.177:3300/0 conn(0x7f125c0cc730 msgr2=0x7f125c0b0670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:51:24.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 --2- 172.21.15.144:0/808223812 >> v2:172.21.15.177:3300/0 conn(0x7f125c0cc730 0x7f125c0b0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f125b7fe700 1 -- 172.21.15.144:0/808223812 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c165270 con 0x7f125c160f00 2023-11-28T11:51:24.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7fc700eab700 1 Processor -- start 2023-11-28T11:51:24.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f0d3f88f700 1 Processor -- start 2023-11-28T11:51:24.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7f125b7fe700 1 --2- 172.21.15.144:0/808223812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160f00 0x7f125c164840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f125c0c9140 tx=0x7f1250009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8863b1547105c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7ff34dffb700 1 -- 172.21.15.144:0/495795531 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340011830 con 0x7ff35006c990 2023-11-28T11:51:24.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7ff34dffb700 1 -- 172.21.15.144:0/495795531 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff340011990 con 0x7ff35006c990 2023-11-28T11:51:24.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7f0d3f88f700 1 -- start start 2023-11-28T11:51:24.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7ff34dffb700 1 -- 172.21.15.144:0/495795531 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340011d20 con 0x7ff35006c990 2023-11-28T11:51:24.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7f125affd700 1 -- 172.21.15.144:0/808223812 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000b870 con 0x7f125c160f00 2023-11-28T11:51:24.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7f125affd700 1 -- 172.21.15.144:0/808223812 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125000b9d0 con 0x7f125c160f00 2023-11-28T11:51:24.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7f125affd700 1 -- 172.21.15.144:0/808223812 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000bd60 con 0x7f125c160f00 2023-11-28T11:51:24.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.713+0000 7fc700eab700 1 -- start start 2023-11-28T11:51:24.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc700eab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6fc0b0d80 0x7fc6fc141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc700eab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6fc159490 0x7fc6fc166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc700eab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc14f430 0x7fc6fc166f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc700eab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc14f8c0 con 0x7fc6fc14f430 2023-11-28T11:51:24.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc700eab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0b2a70 con 0x7fc6fc159490 2023-11-28T11:51:24.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc700eab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0b2d50 con 0x7fc6fc0b0d80 2023-11-28T11:51:24.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7ff355998700 1 -- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff35006c990 msgr2=0x7ff35006ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7ff355998700 1 --2- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff35006c990 0x7ff35006ce10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff340009840 tx=0x7ff340009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:24.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7ff355998700 1 -- 172.21.15.144:0/495795531 shutdown_connections 2023-11-28T11:51:24.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7ff355998700 1 --2- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3500ca240 0x7ff3500ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7ff355998700 1 --2- 172.21.15.144:0/495795531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff35006c990 0x7ff35006ce10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7ff355998700 1 -- 172.21.15.144:0/495795531 >> 172.21.15.144:0/495795531 conn(0x7ff3500b12b0 msgr2=0x7ff3500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7ff355998700 1 -- 172.21.15.144:0/495795531 shutdown_connections 2023-11-28T11:51:24.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7ff355998700 1 -- 172.21.15.144:0/495795531 wait complete. 2023-11-28T11:51:24.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3f88f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0d3806d4d0 0x7f0d380b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3f88f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0d3806b820 0x7f0d3806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3f88f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d380b0f80 con 0x7f0d380b0670 2023-11-28T11:51:24.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3f88f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380ad6e0 con 0x7f0d3806b820 2023-11-28T11:51:24.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3f88f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380aca60 con 0x7f0d3806d4d0 2023-11-28T11:51:24.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3d62b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0d380b0670 0x7f0d380cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51508/0 (socket says 172.21.15.144:51508) 2023-11-28T11:51:24.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3d62b700 1 -- 172.21.15.144:0/639031543 learned_addr learned my addr 172.21.15.144:0/639031543 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:24.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f0d3ce2a700 1 --2- 172.21.15.144:0/639031543 >> v2:172.21.15.177:3300/0 conn(0x7f0d3806d4d0 0x7f0d380b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d3ce2a700 1 -- 172.21.15.144:0/639031543 >> v2:172.21.15.144:3300/0 conn(0x7f0d3806b820 msgr2=0x7f0d3806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d3ce2a700 1 --2- 172.21.15.144:0/639031543 >> v2:172.21.15.144:3300/0 conn(0x7f0d3806b820 0x7f0d3806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d2ffff700 1 -- 172.21.15.144:0/639031543 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176367936 0 0) 0x7f0d380b0f80 con 0x7f0d380b0670 2023-11-28T11:51:24.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d3ce2a700 1 -- 172.21.15.144:0/639031543 >> v1:172.21.15.144:6789/0 conn(0x7f0d380b0670 legacy=0x7f0d380cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d3ce2a700 1 -- 172.21.15.144:0/639031543 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d380acd40 con 0x7f0d3806d4d0 2023-11-28T11:51:24.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d3ce2a700 1 --2- 172.21.15.144:0/639031543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3806d4d0 0x7f0d380b0a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009840 tx=0x7f0d28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c620db2c15ca8a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc6fa59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6fc0b0d80 0x7fc6fc141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc6fad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc14f430 0x7fc6fc166f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc6fad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc14f430 0x7fc6fc166f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34786/0 (socket says 172.21.15.144:34786) 2023-11-28T11:51:24.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc6fad9d700 1 -- 172.21.15.144:0/3215342919 learned_addr learned my addr 172.21.15.144:0/3215342919 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7fc6f9d9b700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6fc159490 0x7fc6fc166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7fc6f9d9b700 1 -- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6fc0b0d80 msgr2=0x7fc6fc141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7fc6f9d9b700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6fc0b0d80 0x7fc6fc141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7fc6f9d9b700 1 -- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc14f430 msgr2=0x7fc6fc166f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7fc6f9d9b700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc14f430 0x7fc6fc166f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7fc6f9d9b700 1 -- 172.21.15.144:0/3215342919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0094a0 con 0x7fc6fc159490 2023-11-28T11:51:24.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7fc6f9d9b700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6fc159490 0x7fc6fc166a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec016000 tx=0x7fc6ec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f6383854700 1 -- 172.21.15.144:0/3040842555 shutdown_connections 2023-11-28T11:51:24.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d2ffff700 1 -- 172.21.15.144:0/639031543 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28011830 con 0x7f0d3806d4d0 2023-11-28T11:51:24.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d2ffff700 1 -- 172.21.15.144:0/639031543 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28011990 con 0x7f0d3806d4d0 2023-11-28T11:51:24.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f0d2ffff700 1 -- 172.21.15.144:0/639031543 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28011d20 con 0x7f0d3806d4d0 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f813d4d5700 1 Processor -- start 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.712+0000 7f813d4d5700 1 -- start start 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f813d4d5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f81380ce520 0x7f81380cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f813d4d5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380b0670 0x7f81380b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f813d4d5700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813806c2f0 con 0x7f81380cacb0 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f813d4d5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81380b1050 con 0x7f81380ce520 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f813d4d5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380cd740 con 0x7f81380b0670 2023-11-28T11:51:24.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f8136ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f81380cacb0 0x7f81380cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51496/0 (socket says 172.21.15.144:51496) 2023-11-28T11:51:24.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f8136ffd700 1 -- 172.21.15.144:0/2214528987 learned_addr learned my addr 172.21.15.144:0/2214528987 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:24.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.714+0000 7f81377fe700 1 --2- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380b0670 0x7f81380b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f812ffff700 1 --2- 172.21.15.144:0/2214528987 >> v2:172.21.15.177:3300/0 conn(0x7f81380ce520 0x7f81380cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 -- 172.21.15.144:0/808223812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160f00 msgr2=0x7f125c164840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 --2- 172.21.15.144:0/808223812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160f00 0x7f125c164840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f125c0c9140 tx=0x7f1250009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 -- 172.21.15.144:0/808223812 shutdown_connections 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 --2- 172.21.15.144:0/808223812 >> v2:172.21.15.177:3300/0 conn(0x7f125c0cc730 0x7f125c0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 --2- 172.21.15.144:0/808223812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160b30 0x7f125c0c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 --2- 172.21.15.144:0/808223812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160f00 0x7f125c164840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f1262323700 1 -- 172.21.15.144:0/808223812 >> 172.21.15.144:0/808223812 conn(0x7f125c0cb670 msgr2=0x7f125c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f0d3f88f700 1 -- 172.21.15.144:0/639031543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3806d4d0 msgr2=0x7f0d380b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/639031543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3806d4d0 0x7f0d380b0a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009840 tx=0x7f0d28009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:24.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0038a0 con 0x7fc6fc159490 2023-11-28T11:51:24.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec01e070 con 0x7fc6fc159490 2023-11-28T11:51:24.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec003c00 con 0x7fc6fc159490 2023-11-28T11:51:24.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7ff355998700 1 Processor -- start 2023-11-28T11:51:24.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f812ffff700 1 -- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380b0670 msgr2=0x7f81380b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f812ffff700 1 --2- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380b0670 0x7f81380b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f812ffff700 1 -- 172.21.15.144:0/2214528987 >> v1:172.21.15.144:6789/0 conn(0x7f81380cacb0 legacy=0x7f81380cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f812ffff700 1 -- 172.21.15.144:0/2214528987 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813806b9c0 con 0x7f81380ce520 2023-11-28T11:51:24.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.716+0000 7f812ffff700 1 --2- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380ce520 0x7f81380cb080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8120009840 tx=0x7f8120009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=294b25bd64098eef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f6383854700 1 -- 172.21.15.144:0/3040842555 wait complete. 2023-11-28T11:51:24.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f6383854700 1 Processor -- start 2023-11-28T11:51:24.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f1262323700 1 -- 172.21.15.144:0/808223812 shutdown_connections 2023-11-28T11:51:24.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7ff355998700 1 -- start start 2023-11-28T11:51:24.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.715+0000 7f81367fc700 1 -- 172.21.15.144:0/2214528987 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960728372 0 0) 0x7f813806c2f0 con 0x7f81380cacb0 2023-11-28T11:51:24.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f81367fc700 1 -- 172.21.15.144:0/2214528987 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120012b80 con 0x7f81380ce520 2023-11-28T11:51:24.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f81367fc700 1 -- 172.21.15.144:0/2214528987 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120012ce0 con 0x7f81380ce520 2023-11-28T11:51:24.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f81367fc700 1 -- 172.21.15.144:0/2214528987 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812001b600 con 0x7f81380ce520 2023-11-28T11:51:24.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f1262323700 1 -- 172.21.15.144:0/808223812 wait complete. 2023-11-28T11:51:24.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 -- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380ce520 msgr2=0x7f81380cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 --2- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380ce520 0x7f81380cb080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8120009840 tx=0x7f8120009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff355998700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff35006c990 0x7ff350154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff355998700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff35006e9f0 0x7ff3501f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff355998700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3500ca240 0x7ff3501f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff355998700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501f6a50 con 0x7ff35006e9f0 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff355998700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501f6f10 con 0x7ff3500ca240 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff355998700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500c5cc0 con 0x7ff35006c990 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff34effd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff35006c990 0x7ff350154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff34effd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff35006c990 0x7ff350154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46886/0 (socket says 172.21.15.144:46886) 2023-11-28T11:51:24.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff34effd700 1 -- 172.21.15.144:0/658493723 learned_addr learned my addr 172.21.15.144:0/658493723 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff34e7fc700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff35006e9f0 0x7ff3501f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7ff34f7fe700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3500ca240 0x7ff3501f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff34f7fe700 1 -- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff35006c990 msgr2=0x7ff350154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff34f7fe700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff35006c990 0x7ff350154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff34f7fe700 1 -- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff35006e9f0 msgr2=0x7ff3501f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff34f7fe700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff35006e9f0 0x7ff3501f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff34f7fe700 1 -- 172.21.15.144:0/658493723 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400094a0 con 0x7ff3500ca240 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff34f7fe700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3500ca240 0x7ff3501f6510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff34800eba0 tx=0x7ff34800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348009a80 con 0x7ff3500ca240 2023-11-28T11:51:24.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f1262323700 1 Processor -- start 2023-11-28T11:51:24.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f1262323700 1 -- start start 2023-11-28T11:51:24.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f9d3b248700 1 Processor -- start 2023-11-28T11:51:24.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f0d3f88f700 1 -- 172.21.15.144:0/639031543 shutdown_connections 2023-11-28T11:51:24.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/639031543 >> v2:172.21.15.144:3300/0 conn(0x7f0d3806b820 0x7f0d3806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/639031543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3806d4d0 0x7f0d380b0a40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f0d3f88f700 1 -- 172.21.15.144:0/639031543 >> 172.21.15.144:0/639031543 conn(0x7f0d380cb670 msgr2=0x7f0d380cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc0c9f40 con 0x7fc6fc159490 2023-11-28T11:51:24.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc0cd760 con 0x7fc6fc159490 2023-11-28T11:51:24.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.717+0000 7f6383854700 1 -- start start 2023-11-28T11:51:24.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6383854700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0b0670 0x7f637c0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6383854700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cacb0 0x7f637c161e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6383854700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f637c0cde10 0x7f637c162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6383854700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1612d0 con 0x7f637c0b0670 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6383854700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c160470 con 0x7f637c0cde10 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6383854700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c160750 con 0x7f637c0cacb0 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6380def700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cacb0 0x7f637c161e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6380def700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cacb0 0x7f637c161e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46888/0 (socket says 172.21.15.144:46888) 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6380def700 1 -- 172.21.15.144:0/3157446220 learned_addr learned my addr 172.21.15.144:0/3157446220 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f6381df1700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f637c0cde10 0x7f637c162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f63815f0700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0b0670 0x7f637c0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f63815f0700 1 -- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cacb0 msgr2=0x7f637c161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f63815f0700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cacb0 0x7f637c161e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f63815f0700 1 -- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f637c0cde10 msgr2=0x7f637c162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f63815f0700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f637c0cde10 0x7f637c162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f63815f0700 1 -- 172.21.15.144:0/3157446220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637000d9f0 con 0x7f637c0b0670 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f63815f0700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0b0670 0x7f637c0c4c30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6370019000 tx=0x7f6370004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000b070 con 0x7f637c0b0670 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63700036a0 con 0x7f637c0b0670 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370003a50 con 0x7f637c0b0670 2023-11-28T11:51:24.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c06bf80 con 0x7f637c0b0670 2023-11-28T11:51:24.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 -- 172.21.15.144:0/2214528987 shutdown_connections 2023-11-28T11:51:24.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 --2- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380b0670 0x7f81380b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 --2- 172.21.15.144:0/2214528987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380ce520 0x7f81380cb080 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 -- 172.21.15.144:0/2214528987 >> 172.21.15.144:0/2214528987 conn(0x7f81380cb670 msgr2=0x7f81380cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 -- 172.21.15.144:0/2214528987 shutdown_connections 2023-11-28T11:51:24.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c06bca0 con 0x7f637c0b0670 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f1262323700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125c0cc730 0x7f125c0c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f1262323700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 0x7f125c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f1262323700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160f00 0x7f125c15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f1262323700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1fac80 con 0x7f125c160b30 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f1262323700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c0b28d0 con 0x7f125c0cc730 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f1262323700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1fa100 con 0x7f125c160f00 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f125b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 0x7f125c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f125b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 0x7f125c0b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34816/0 (socket says 172.21.15.144:34816) 2023-11-28T11:51:24.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f125b7fe700 1 -- 172.21.15.144:0/1299532550 learned_addr learned my addr 172.21.15.144:0/1299532550 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35006a810 con 0x7ff3500ca240 2023-11-28T11:51:24.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35006a530 con 0x7ff3500ca240 2023-11-28T11:51:24.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35006aaf0 con 0x7ff3500ca240 2023-11-28T11:51:24.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34800f040 con 0x7ff3500ca240 2023-11-28T11:51:24.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800cc40 con 0x7ff3500ca240 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f12608c0700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160f00 0x7f125c15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f125b7fe700 1 -- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160f00 msgr2=0x7f125c15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f125b7fe700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160f00 0x7f125c15ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f125b7fe700 1 -- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125c0cc730 msgr2=0x7f125c0c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f125b7fe700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125c0cc730 0x7f125c0c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f125b7fe700 1 -- 172.21.15.144:0/1299532550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250009550 con 0x7f125c160b30 2023-11-28T11:51:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f125b7fe700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 0x7f125c0b4070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1250013000 tx=0x7f1250004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250004460 con 0x7f125c160b30 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c0ccbd0 con 0x7f125c160b30 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c165550 con 0x7f125c160b30 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125001e070 con 0x7f125c160b30 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500038a0 con 0x7f125c160b30 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.719+0000 7f9d3b248700 1 -- start start 2023-11-28T11:51:24.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f9d3b248700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d34158890 0x7f9d340ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f9d3b248700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f9d3b248700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3415c490 con 0x7f9d340b0bb0 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f9d3b248700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340b1030 con 0x7f9d34158890 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7f9d3b248700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340ce650 con 0x7f9d3406d4f0 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d38fe4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d34158890 0x7f9d340ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d38fe4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9d34158890 0x7f9d340ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:37094/0 (socket says 172.21.15.144:37094) 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d38fe4700 1 -- 172.21.15.144:0/1728882024 learned_addr learned my addr 172.21.15.144:0/1728882024 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d33fff700 1 --2- 172.21.15.144:0/1728882024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d33fff700 1 -- 172.21.15.144:0/1728882024 >> v2:172.21.15.177:3300/0 conn(0x7f9d34158890 msgr2=0x7f9d340ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d33fff700 1 --2- 172.21.15.144:0/1728882024 >> v2:172.21.15.177:3300/0 conn(0x7f9d34158890 0x7f9d340ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d33fff700 1 -- 172.21.15.144:0/1728882024 >> v1:172.21.15.144:6789/0 conn(0x7f9d340b0bb0 legacy=0x7f9d340cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d33fff700 1 -- 172.21.15.144:0/1728882024 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3415c730 con 0x7f9d3406d4f0 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7f9d33fff700 1 --2- 172.21.15.144:0/1728882024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340b0670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009e10 tx=0x7f9d2c013b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac78dbe1b3a4159 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.721+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc0cdc80 con 0x7fc6fc159490 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6ec00f050 con 0x7fc6fc159490 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7fc6eaffd700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6dc0683c0 0x7fc6dc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6ec086cd0 con 0x7fc6fc159490 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.722+0000 7fc6fa59c700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6dc0683c0 0x7fc6dc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.718+0000 7f813d4d5700 1 -- 172.21.15.144:0/2214528987 wait complete. 2023-11-28T11:51:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.720+0000 7f813d4d5700 1 Processor -- start 2023-11-28T11:51:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f9d337fe700 1 -- 172.21.15.144:0/1728882024 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c003a20 con 0x7f9d3406d4f0 2023-11-28T11:51:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f9d337fe700 1 -- 172.21.15.144:0/1728882024 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c003b80 con 0x7f9d3406d4f0 2023-11-28T11:51:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f9d337fe700 1 -- 172.21.15.144:0/1728882024 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004030 con 0x7f9d3406d4f0 2023-11-28T11:51:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7fc6fa59c700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6dc0683c0 0x7fc6dc06a870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00011e0 tx=0x7fc6f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6370003bb0 con 0x7f637c0b0670 2023-11-28T11:51:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1248005320 con 0x7f125c160b30 2023-11-28T11:51:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f636dffb700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6368068450 0x7f636806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6370086720 con 0x7f637c0b0670 2023-11-28T11:51:24.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f6380def700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6368068450 0x7f636806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f125000f050 con 0x7f125c160b30 2023-11-28T11:51:24.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f6380def700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6368068450 0x7f636806a900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f637c071f00 tx=0x7f637800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f9d3b248700 1 -- 172.21.15.144:0/1728882024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 msgr2=0x7f9d340b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f9d3b248700 1 --2- 172.21.15.144:0/1728882024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340b0670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009e10 tx=0x7f9d2c013b40 comp rx=0 tx=0).stop 2023-11-28T11:51:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f9d3b248700 1 -- 172.21.15.144:0/1728882024 shutdown_connections 2023-11-28T11:51:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f9d3b248700 1 --2- 172.21.15.144:0/1728882024 >> v2:172.21.15.177:3300/0 conn(0x7f9d34158890 0x7f9d340ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f9d3b248700 1 --2- 172.21.15.144:0/1728882024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340b0670 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f9d3b248700 1 -- 172.21.15.144:0/1728882024 >> 172.21.15.144:0/1728882024 conn(0x7f9d340cb670 msgr2=0x7f9d340cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f1258ff9700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1244068460 0x7f124406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f125bfff700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1244068460 0x7f124406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1250020070 con 0x7f125c160b30 2023-11-28T11:51:24.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.723+0000 7f813d4d5700 1 -- start start 2023-11-28T11:51:24.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f813d4d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81380b0670 0x7f8138206ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f813d4d5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 0x7f81380c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f813d4d5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380ce520 0x7f813816a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f813d4d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380c3d90 con 0x7f81380b0670 2023-11-28T11:51:24.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f813d4d5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81380c4070 con 0x7f81380cacb0 2023-11-28T11:51:24.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.724+0000 7f813d4d5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81380c4350 con 0x7f81380ce520 2023-11-28T11:51:24.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f125bfff700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1244068460 0x7f124406a910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c009770 tx=0x7f124c006cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f0d3f88f700 1 -- 172.21.15.144:0/639031543 shutdown_connections 2023-11-28T11:51:24.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f0d3f88f700 1 -- 172.21.15.144:0/639031543 wait complete. 2023-11-28T11:51:24.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6ec08c020 con 0x7fc6fc159490 2023-11-28T11:51:24.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f0d3f88f700 1 Processor -- start 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 0x7f81380c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 0x7f81380c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:37108/0 (socket says 172.21.15.144:37108) 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 -- 172.21.15.144:0/3050966542 learned_addr learned my addr 172.21.15.144:0/3050966542 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f81377fe700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380ce520 0x7f813816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 -- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380ce520 msgr2=0x7f813816a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380ce520 0x7f813816a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 -- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81380b0670 msgr2=0x7f8138206ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81380b0670 0x7f8138206ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 -- 172.21.15.144:0/3050966542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200094a0 con 0x7f81380cacb0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.725+0000 7f812ffff700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 0x7f81380c33f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8120009fd0 tx=0x7f8120004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120012510 con 0x7f81380cacb0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81200036a0 con 0x7f81380cacb0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813806bf80 con 0x7f81380cacb0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120003a50 con 0x7f81380cacb0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813806bca0 con 0x7f81380cacb0 2023-11-28T11:51:24.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.727+0000 7f0d3f88f700 1 -- start start 2023-11-28T11:51:24.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813806c780 con 0x7f81380cacb0 2023-11-28T11:51:24.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff348014480 con 0x7ff3500ca240 2023-11-28T11:51:24.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7ff337fff700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff338068450 0x7ff33806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7ff34effd700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff338068450 0x7ff33806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff348086d50 con 0x7ff3500ca240 2023-11-28T11:51:24.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3f88f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 0x7f0d380c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3f88f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3806d4d0 0x7f0d381f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3f88f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d380b0670 0x7f0d381f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3f88f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380c55b0 con 0x7f0d3806d4d0 2023-11-28T11:51:24.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3f88f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380c4da0 con 0x7f0d380b0670 2023-11-28T11:51:24.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3f88f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d38159ef0 con 0x7f0d3806b820 2023-11-28T11:51:24.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3d62b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 0x7f0d380c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3d62b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 0x7f0d380c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46922/0 (socket says 172.21.15.144:46922) 2023-11-28T11:51:24.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.728+0000 7f0d3d62b700 1 -- 172.21.15.144:0/398785378 learned_addr learned my addr 172.21.15.144:0/398785378 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7ff34effd700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff338068450 0x7ff33806a900 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff344005910 tx=0x7ff344009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3d62b700 1 -- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d380b0670 msgr2=0x7f0d381f6310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:51:24.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3d62b700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d380b0670 0x7f0d381f6310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3d62b700 1 -- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3806d4d0 msgr2=0x7f0d381f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3d62b700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3806d4d0 0x7f0d381f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3d62b700 1 -- 172.21.15.144:0/398785378 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280094a0 con 0x7f0d3806b820 2023-11-28T11:51:24.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3d62b700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 0x7f0d380c2690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c520 tx=0x7f0d3400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34004c80 con 0x7f0d3806b820 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34004e20 con 0x7f0d3806b820 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d380ad300 con 0x7f0d3806b820 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34007310 con 0x7f0d3806b820 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d380ad020 con 0x7f0d3806b820 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.729+0000 7f9d3b248700 1 -- 172.21.15.144:0/1728882024 shutdown_connections 2023-11-28T11:51:24.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.730+0000 7f9d3b248700 1 -- 172.21.15.144:0/1728882024 wait complete. 2023-11-28T11:51:24.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.731+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d3401a020 con 0x7f0d3806b820 2023-11-28T11:51:24.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.731+0000 7f0d2dffb700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d1c0682c0 0x7f0d1c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.731+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d34085840 con 0x7f0d3806b820 2023-11-28T11:51:24.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.731+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff348053540 con 0x7ff3500ca240 2023-11-28T11:51:24.779 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.731+0000 7f0d3ce2a700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d1c0682c0 0x7f0d1c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.779 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.731+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3815caf0 con 0x7f0d3806b820 2023-11-28T11:51:24.779 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.732+0000 7f9d3b248700 1 Processor -- start 2023-11-28T11:51:24.779 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.732+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12500539a0 con 0x7f125c160b30 2023-11-28T11:51:24.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c06c760 con 0x7f637c0b0670 2023-11-28T11:51:24.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f812002a030 con 0x7f81380cacb0 2023-11-28T11:51:24.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f0d3ce2a700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d1c0682c0 0x7f0d1c06a770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d380afa20 tx=0x7f0d28010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.732+0000 7f9d3b248700 1 -- start start 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f9d3b248700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f9d3b248700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d340b0bb0 0x7f9d34146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f9d3b248700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 0x7f9d340b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f9d3b248700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341473b0 con 0x7f9d34158890 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f9d3b248700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340c3020 con 0x7f9d340b0bb0 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f9d3b248700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d340c3270 con 0x7f9d3406d4f0 2023-11-28T11:51:24.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 0x7f9d340b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 0x7f9d340b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34848/0 (socket says 172.21.15.144:34848) 2023-11-28T11:51:24.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 -- 172.21.15.144:0/3499628917 learned_addr learned my addr 172.21.15.144:0/3499628917 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:24.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d33fff700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d340b0bb0 0x7f9d34146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d38fe4700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.733+0000 7f812f7fe700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8118068320 0x7f811806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f81200873d0 con 0x7f81380cacb0 2023-11-28T11:51:24.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f8136ffd700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8118068320 0x7f811806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.786 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 -- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 msgr2=0x7f9d340c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.786 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.786 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 -- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d340b0bb0 msgr2=0x7f9d34146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:24.786 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d340b0bb0 0x7f9d34146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:24.786 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.734+0000 7f9d397e5700 1 -- 172.21.15.144:0/3499628917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009a70 con 0x7f9d34158890 2023-11-28T11:51:24.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.735+0000 7f8136ffd700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8118068320 0x7f811806a7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8128000c00 tx=0x7f8128007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.735+0000 7f9d397e5700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 0x7f9d340b1a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c490 tx=0x7f9d24001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.736+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8120053cd0 con 0x7f81380cacb0 2023-11-28T11:51:24.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.737+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d340520a0 con 0x7f0d3806b820 2023-11-28T11:51:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.740+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240054e0 con 0x7f9d34158890 2023-11-28T11:51:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.740+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24003680 con 0x7f9d34158890 2023-11-28T11:51:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.740+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003a30 con 0x7f9d34158890 2023-11-28T11:51:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.740+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3415ccf0 con 0x7f9d34158890 2023-11-28T11:51:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.740+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3415ca10 con 0x7f9d34158890 2023-11-28T11:51:24.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.743+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9d24019030 con 0x7f9d34158890 2023-11-28T11:51:24.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.743+0000 7f9d317fa700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9d20068340 0x7f9d2006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:24.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.744+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6370052f80 con 0x7f637c0b0670 2023-11-28T11:51:24.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.744+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d24084f90 con 0x7f9d34158890 2023-11-28T11:51:24.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.744+0000 7f9d38fe4700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9d20068340 0x7f9d2006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:24.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.744+0000 7f9d38fe4700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9d20068340 0x7f9d2006a7f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d28004610 tx=0x7f9d2800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:24.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.750+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d340c9370 con 0x7f9d34158890 2023-11-28T11:51:24.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:24.759+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d24051810 con 0x7f9d34158890 2023-11-28T11:51:25.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.059+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc15c0745c0 con 0x7fc15c06c660 2023-11-28T11:51:25.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.063+0000 7fc1437fe700 1 -- 172.21.15.144:0/3285894769 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc150015090 con 0x7fc15c06c660 2023-11-28T11:51:25.066 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1480683c0 msgr2=0x7fc14806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1480683c0 0x7fc14806a870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0090b0 tx=0x7fc14c010000 comp rx=0 tx=0).stop 2023-11-28T11:51:25.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 msgr2=0x7fc15c0cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 0x7fc15c0cc880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc15000b580 tx=0x7fc15000b890 comp rx=0 tx=0).stop 2023-11-28T11:51:25.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 shutdown_connections 2023-11-28T11:51:25.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1480683c0 0x7fc14806a870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc15c164c30 0x7fc15c1510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc15c06c660 0x7fc15c0cc880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 --2- 172.21.15.144:0/3285894769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c0b0750 0x7fc15c0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.071+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 >> 172.21.15.144:0/3285894769 conn(0x7fc15c06aeb0 msgr2=0x7fc15c0cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.072+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 shutdown_connections 2023-11-28T11:51:25.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.072+0000 7fc161528700 1 -- 172.21.15.144:0/3285894769 wait complete. 2023-11-28T11:51:25.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2023-11-28T11:51:25.091 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.116+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff3500cd5b0 con 0x7ff3500ca240 2023-11-28T11:51:25.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.117+0000 7ff337fff700 1 -- 172.21.15.144:0/658493723 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff348018090 con 0x7ff3500ca240 2023-11-28T11:51:25.118 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.120+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff338068450 msgr2=0x7ff33806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.120+0000 7ff355998700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff338068450 0x7ff33806a900 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff344005910 tx=0x7ff344009040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.120+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3500ca240 msgr2=0x7ff3501f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.120+0000 7ff355998700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3500ca240 0x7ff3501f6510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff34800eba0 tx=0x7ff34800eeb0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.121+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 shutdown_connections 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.121+0000 7ff355998700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff338068450 0x7ff33806a900 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.121+0000 7ff355998700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3500ca240 0x7ff3501f6510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.121+0000 7ff355998700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff35006c990 0x7ff350154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.121+0000 7ff355998700 1 --2- 172.21.15.144:0/658493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff35006e9f0 0x7ff3501f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.121+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 >> 172.21.15.144:0/658493723 conn(0x7ff3500b12b0 msgr2=0x7ff350065f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.122+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9d340745c0 con 0x7f9d34158890 2023-11-28T11:51:25.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.122+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 shutdown_connections 2023-11-28T11:51:25.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.122+0000 7ff355998700 1 -- 172.21.15.144:0/658493723 wait complete. 2023-11-28T11:51:25.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.122+0000 7f9d317fa700 1 -- 172.21.15.144:0/3499628917 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d24017070 con 0x7f9d34158890 2023-11-28T11:51:25.124 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.125+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0d380745c0 con 0x7f0d3806b820 2023-11-28T11:51:25.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.125+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f81380745c0 con 0x7f81380cacb0 2023-11-28T11:51:25.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.125+0000 7f812f7fe700 1 -- 172.21.15.144:0/3050966542 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f81200572f0 con 0x7f81380cacb0 2023-11-28T11:51:25.126 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.127 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.126+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f12480055d0 con 0x7f125c160b30 2023-11-28T11:51:25.127 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.127+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc6fc0cc010 con 0x7fc6fc159490 2023-11-28T11:51:25.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.127+0000 7fc6eaffd700 1 -- 172.21.15.144:0/3215342919 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6ec015020 con 0x7fc6fc159490 2023-11-28T11:51:25.128 INFO:teuthology.orchestra.run.smithi144.stdout:60129542149 2023-11-28T11:51:25.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.128+0000 7f0d2dffb700 1 -- 172.21.15.144:0/398785378 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d340556c0 con 0x7f0d3806b820 2023-11-28T11:51:25.128 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.129+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9d20068340 msgr2=0x7f9d2006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.129+0000 7f9d3b248700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9d20068340 0x7f9d2006a7f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d28004610 tx=0x7f9d2800d040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.129+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 msgr2=0x7f9d340b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.129+0000 7f9d3b248700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 0x7f9d340b1a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c490 tx=0x7f9d24001e70 comp rx=0 tx=0).stop 2023-11-28T11:51:25.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.129+0000 7f1258ff9700 1 -- 172.21.15.144:0/1299532550 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1250056fc0 con 0x7f125c160b30 2023-11-28T11:51:25.130 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8118068320 msgr2=0x7f811806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8118068320 0x7f811806a7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8128000c00 tx=0x7f8128007990 comp rx=0 tx=0).stop 2023-11-28T11:51:25.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 msgr2=0x7f81380c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 0x7f81380c33f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8120009fd0 tx=0x7f8120004750 comp rx=0 tx=0).stop 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 shutdown_connections 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81380b0670 0x7f8138206ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8118068320 0x7f811806a7d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81380ce520 0x7f813816a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 --2- 172.21.15.144:0/3050966542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81380cacb0 0x7f81380c33f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 >> 172.21.15.144:0/3050966542 conn(0x7f81380cb670 msgr2=0x7f8138070930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 shutdown_connections 2023-11-28T11:51:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f813d4d5700 1 -- 172.21.15.144:0/3050966542 wait complete. 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6dc0683c0 msgr2=0x7fc6dc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6dc0683c0 0x7fc6dc06a870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00011e0 tx=0x7fc6f0009040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6fc159490 msgr2=0x7fc6fc166a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6fc159490 0x7fc6fc166a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec016000 tx=0x7fc6ec004750 comp rx=0 tx=0).stop 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 shutdown_connections 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6fc14f430 0x7fc6fc166f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6dc0683c0 0x7fc6dc06a870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6fc0b0d80 0x7fc6fc141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 --2- 172.21.15.144:0/3215342919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6fc159490 0x7fc6fc166a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 >> 172.21.15.144:0/3215342919 conn(0x7fc6fc0cb670 msgr2=0x7fc6fc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 shutdown_connections 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f9d3b248700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d34158890 0x7f9d340b1a20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f9d3b248700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9d20068340 0x7f9d2006a7f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f9d3b248700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d3406d4f0 0x7f9d340c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f9d3b248700 1 --2- 172.21.15.144:0/3499628917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d340b0bb0 0x7f9d34146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.130+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 >> 172.21.15.144:0/3499628917 conn(0x7f9d340cb670 msgr2=0x7f9d340ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d1c0682c0 msgr2=0x7f0d1c06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d1c0682c0 0x7f0d1c06a770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0d380afa20 tx=0x7f0d28010000 comp rx=0 tx=0).stop 2023-11-28T11:51:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 msgr2=0x7f0d380c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 0x7f0d380c2690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c520 tx=0x7f0d3400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 shutdown_connections 2023-11-28T11:51:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7fc700eab700 1 -- 172.21.15.144:0/3215342919 wait complete. 2023-11-28T11:51:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 shutdown_connections 2023-11-28T11:51:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d1c0682c0 0x7f0d1c06a770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d380b0670 0x7f0d381f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3806b820 0x7f0d380c2690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 --2- 172.21.15.144:0/398785378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3806d4d0 0x7f0d381f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 >> 172.21.15.144:0/398785378 conn(0x7f0d380cb670 msgr2=0x7f0d380ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 shutdown_connections 2023-11-28T11:51:25.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.131+0000 7f0d3f88f700 1 -- 172.21.15.144:0/398785378 wait complete. 2023-11-28T11:51:25.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.132+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 shutdown_connections 2023-11-28T11:51:25.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.132+0000 7f9d3b248700 1 -- 172.21.15.144:0/3499628917 wait complete. 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1244068460 msgr2=0x7f124406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1244068460 0x7f124406a910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c009770 tx=0x7f124c006cd0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 msgr2=0x7f125c0b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 0x7f125c0b4070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1250013000 tx=0x7f1250004750 comp rx=0 tx=0).stop 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 shutdown_connections 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1244068460 0x7f124406a910 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f124c009770 tx=0x7f124c006cd0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125c160f00 0x7f125c15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125c0cc730 0x7f125c0c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 --2- 172.21.15.144:0/1299532550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125c160b30 0x7f125c0b4070 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1250013000 tx=0x7f1250004750 comp rx=0 tx=0).stop 2023-11-28T11:51:25.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.133+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 >> 172.21.15.144:0/1299532550 conn(0x7f125c0cb670 msgr2=0x7f125c0ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.139+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 shutdown_connections 2023-11-28T11:51:25.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.140+0000 7f1262323700 1 -- 172.21.15.144:0/1299532550 wait complete. 2023-11-28T11:51:25.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2023-11-28T11:51:25.142 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.147+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f637c0745c0 con 0x7f637c0b0670 2023-11-28T11:51:25.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.148+0000 7f636dffb700 1 -- 172.21.15.144:0/3157446220 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63700565a0 con 0x7f637c0b0670 2023-11-28T11:51:25.151 INFO:teuthology.orchestra.run.smithi144.stdout:60129542150 2023-11-28T11:51:25.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6368068450 msgr2=0x7f636806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6368068450 0x7f636806a900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f637c071f00 tx=0x7f637800d040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0b0670 msgr2=0x7f637c0c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0b0670 0x7f637c0c4c30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6370019000 tx=0x7f6370004750 comp rx=0 tx=0).stop 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 shutdown_connections 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f637c0b0670 0x7f637c0c4c30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6368068450 0x7f636806a900 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f637c071f00 tx=0x7f637800d040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f637c0cde10 0x7f637c162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 --2- 172.21.15.144:0/3157446220 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f637c0cacb0 0x7f637c161e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.152+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 >> 172.21.15.144:0/3157446220 conn(0x7f637c0cb670 msgr2=0x7f637c169020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2023-11-28T11:51:25.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.153+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 shutdown_connections 2023-11-28T11:51:25.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.153+0000 7f6383854700 1 -- 172.21.15.144:0/3157446220 wait complete. 2023-11-28T11:51:25.155 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2023-11-28T11:51:25.157 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2023-11-28T11:51:25.158 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2023-11-28T11:51:25.159 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2023-11-28T11:51:25.163 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2023-11-28T11:51:25.169 DEBUG:teuthology.parallel:result is None 2023-11-28T11:51:25.169 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:51:25.169 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:51:25.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.327+0000 7f5fcce31700 1 Processor -- start 2023-11-28T11:51:25.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.327+0000 7f5fcce31700 1 -- start start 2023-11-28T11:51:25.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fcce31700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5fc806c990 0x7f5fc806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fcce31700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 0x7f5fc80ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fcce31700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc806ae60 con 0x7f5fc806e9f0 2023-11-28T11:51:25.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fcce31700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc806d350 con 0x7f5fc806c990 2023-11-28T11:51:25.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fcce31700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80cabc0 con 0x7f5fc80ca240 2023-11-28T11:51:25.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fc6d9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 0x7f5fc80ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fc5d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5fc806c990 0x7f5fc806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fc6d9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 0x7f5fc80ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46942/0 (socket says 172.21.15.144:46942) 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.328+0000 7f5fc6d9d700 1 -- 172.21.15.144:0/1672646437 learned_addr learned my addr 172.21.15.144:0/1672646437 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc6d9d700 1 -- 172.21.15.144:0/1672646437 >> v2:172.21.15.177:3300/0 conn(0x7f5fc806c990 msgr2=0x7f5fc806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc6d9d700 1 --2- 172.21.15.144:0/1672646437 >> v2:172.21.15.177:3300/0 conn(0x7f5fc806c990 0x7f5fc806ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc6d9d700 1 -- 172.21.15.144:0/1672646437 >> v1:172.21.15.177:6789/0 conn(0x7f5fc806e9f0 legacy=0x7f5fc806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc6d9d700 1 -- 172.21.15.144:0/1672646437 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc806a270 con 0x7f5fc80ca240 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc5d9b700 1 --2- 172.21.15.144:0/1672646437 >> v2:172.21.15.177:3300/0 conn(0x7f5fc806c990 0x7f5fc806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc6d9d700 1 --2- 172.21.15.144:0/1672646437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 0x7f5fc80ca680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009e50 tx=0x7f5fc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb4a3043ba4d916 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:25.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc559a700 1 -- 172.21.15.144:0/1672646437 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000e070 con 0x7f5fc80ca240 2023-11-28T11:51:25.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc559a700 1 -- 172.21.15.144:0/1672646437 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000b910 con 0x7f5fc80ca240 2023-11-28T11:51:25.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.329+0000 7f5fc559a700 1 -- 172.21.15.144:0/1672646437 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000bcc0 con 0x7f5fc80ca240 2023-11-28T11:51:25.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 -- 172.21.15.144:0/1672646437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 msgr2=0x7f5fc80ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 --2- 172.21.15.144:0/1672646437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 0x7f5fc80ca680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009e50 tx=0x7f5fc000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:25.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 -- 172.21.15.144:0/1672646437 shutdown_connections 2023-11-28T11:51:25.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 --2- 172.21.15.144:0/1672646437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc80ca240 0x7f5fc80ca680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 --2- 172.21.15.144:0/1672646437 >> v2:172.21.15.177:3300/0 conn(0x7f5fc806c990 0x7f5fc806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 -- 172.21.15.144:0/1672646437 >> 172.21.15.144:0/1672646437 conn(0x7f5fc80b12b0 msgr2=0x7f5fc80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 -- 172.21.15.144:0/1672646437 shutdown_connections 2023-11-28T11:51:25.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.330+0000 7f5fcce31700 1 -- 172.21.15.144:0/1672646437 wait complete. 2023-11-28T11:51:25.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.331+0000 7f5fcce31700 1 Processor -- start 2023-11-28T11:51:25.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.331+0000 7f5fcce31700 1 -- start start 2023-11-28T11:51:25.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc806c990 0x7f5fc8164d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 0x7f5fc814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fc80ca240 0x7f5fc80b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80ae2d0 con 0x7f5fc806e9f0 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc8155230 con 0x7f5fc80ca240 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc80ae570 con 0x7f5fc806c990 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 0x7f5fc814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 0x7f5fc814d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34866/0 (socket says 172.21.15.144:34866) 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 -- 172.21.15.144:0/3513499725 learned_addr learned my addr 172.21.15.144:0/3513499725 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc659c700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc806c990 0x7f5fc8164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc6d9d700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fc80ca240 0x7f5fc80b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 -- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc806c990 msgr2=0x7f5fc8164d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc806c990 0x7f5fc8164d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 -- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fc80ca240 msgr2=0x7f5fc80b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fc80ca240 0x7f5fc80b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 -- 172.21.15.144:0/3513499725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80094a0 con 0x7f5fc806e9f0 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fc5d9b700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 0x7f5fc814d4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800ebe0 tx=0x7f5fb800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800c850 con 0x7f5fc806e9f0 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb800c9b0 con 0x7f5fc806e9f0 2023-11-28T11:51:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.332+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc0009b30 con 0x7f5fc806e9f0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.333+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800cd40 con 0x7f5fc806e9f0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.333+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc806a810 con 0x7f5fc806e9f0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.334+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc806a530 con 0x7f5fc806e9f0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.334+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fb8019030 con 0x7f5fc806e9f0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.334+0000 7f5fb6ffd700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5fa80682f0 0x7f5fa806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.334+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5fb8085c00 con 0x7f5fc806e9f0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.334+0000 7f5fc659c700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5fa80682f0 0x7f5fa806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.335+0000 7f5fc659c700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5fa80682f0 0x7f5fa806a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80b2340 tx=0x7f5fbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:25.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.339+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5fb808b020 con 0x7f5fc806e9f0 2023-11-28T11:51:25.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.517+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fc80ae6e0 con 0x7f5fa80682f0 2023-11-28T11:51:25.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.525+0000 7f5fb6ffd700 1 -- 172.21.15.144:0/3513499725 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286772 (secure 0 0 0) 0x7f5fc80ae6e0 con 0x7f5fa80682f0 2023-11-28T11:51:25.529 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5fa80682f0 msgr2=0x7f5fa806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5fa80682f0 0x7f5fa806a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80b2340 tx=0x7f5fbc009040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 msgr2=0x7f5fc814d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 0x7f5fc814d4e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800ebe0 tx=0x7f5fb800eef0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 shutdown_connections 2023-11-28T11:51:25.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5fa80682f0 0x7f5fa806a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fc80ca240 0x7f5fc80b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc806c990 0x7f5fc8164d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 --2- 172.21.15.144:0/3513499725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fc806e9f0 0x7f5fc814d4e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 >> 172.21.15.144:0/3513499725 conn(0x7f5fc80b12b0 msgr2=0x7f5fc806b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 shutdown_connections 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.528+0000 7f5fcce31700 1 -- 172.21.15.144:0/3513499725 wait complete. 2023-11-28T11:51:25.532 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:51:25.541 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T11:51:24.484331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66080,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908640,"statfs":{"total":773094113280,"available":773026447360,"internally_reserved":0,"allocated":3047424,"data_stored":1845464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003067"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092580+0000","last_change":"2023-11-28T11:51:16.886968+0000","last_active":"2023-11-28T11:51:19.092580+0000","last_peered":"2023-11-28T11:51:19.092580+0000","last_clean":"2023-11-28T11:51:19.092580+0000","last_became_active":"2023-11-28T11:51:16.885939+0000","last_became_peered":"2023-11-28T11:51:16.885939+0000","last_unstale":"2023-11-28T11:51:19.092580+0000","last_undegraded":"2023-11-28T11:51:19.092580+0000","last_fullsized":"2023-11-28T11:51:19.092580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:14.596675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879462+0000","last_change":"2023-11-28T11:51:16.887419+0000","last_active":"2023-11-28T11:51:17.879462+0000","last_peered":"2023-11-28T11:51:17.879462+0000","last_clean":"2023-11-28T11:51:17.879462+0000","last_became_active":"2023-11-28T11:51:16.887120+0000","last_became_peered":"2023-11-28T11:51:16.887120+0000","last_unstale":"2023-11-28T11:51:17.879462+0000","last_undegraded":"2023-11-28T11:51:17.879462+0000","last_fullsized":"2023-11-28T11:51:17.879462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881469+0000","last_change":"2023-11-28T11:51:16.899556+0000","last_active":"2023-11-28T11:51:17.881469+0000","last_peered":"2023-11-28T11:51:17.881469+0000","last_clean":"2023-11-28T11:51:17.881469+0000","last_became_active":"2023-11-28T11:51:16.899088+0000","last_became_peered":"2023-11-28T11:51:16.899088+0000","last_unstale":"2023-11-28T11:51:17.881469+0000","last_undegraded":"2023-11-28T11:51:17.881469+0000","last_fullsized":"2023-11-28T11:51:17.881469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:23.190053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093312+0000","last_change":"2023-11-28T11:51:16.898501+0000","last_active":"2023-11-28T11:51:19.093312+0000","last_peered":"2023-11-28T11:51:19.093312+0000","last_clean":"2023-11-28T11:51:19.093312+0000","last_became_active":"2023-11-28T11:51:16.898263+0000","last_became_peered":"2023-11-28T11:51:16.898263+0000","last_unstale":"2023-11-28T11:51:19.093312+0000","last_undegraded":"2023-11-28T11:51:19.093312+0000","last_fullsized":"2023-11-28T11:51:19.093312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093023+0000","last_change":"2023-11-28T11:51:16.885205+0000","last_active":"2023-11-28T11:51:19.093023+0000","last_peered":"2023-11-28T11:51:19.093023+0000","last_clean":"2023-11-28T11:51:19.093023+0000","last_became_active":"2023-11-28T11:51:16.884964+0000","last_became_peered":"2023-11-28T11:51:16.884964+0000","last_unstale":"2023-11-28T11:51:19.093023+0000","last_undegraded":"2023-11-28T11:51:19.093023+0000","last_fullsized":"2023-11-28T11:51:19.093023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:36.307669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881261+0000","last_change":"2023-11-28T11:51:16.899331+0000","last_active":"2023-11-28T11:51:17.881261+0000","last_peered":"2023-11-28T11:51:17.881261+0000","last_clean":"2023-11-28T11:51:17.881261+0000","last_became_active":"2023-11-28T11:51:16.899026+0000","last_became_peered":"2023-11-28T11:51:16.899026+0000","last_unstale":"2023-11-28T11:51:17.881261+0000","last_undegraded":"2023-11-28T11:51:17.881261+0000","last_fullsized":"2023-11-28T11:51:17.881261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:40.625446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093069+0000","last_change":"2023-11-28T11:51:16.895279+0000","last_active":"2023-11-28T11:51:19.093069+0000","last_peered":"2023-11-28T11:51:19.093069+0000","last_clean":"2023-11-28T11:51:19.093069+0000","last_became_active":"2023-11-28T11:51:16.894949+0000","last_became_peered":"2023-11-28T11:51:16.894949+0000","last_unstale":"2023-11-28T11:51:19.093069+0000","last_undegraded":"2023-11-28T11:51:19.093069+0000","last_fullsized":"2023-11-28T11:51:19.093069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.882353+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.882010+0000","last_became_peered":"2023-11-28T11:51:16.882010+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093331+0000","last_change":"2023-11-28T11:51:16.885987+0000","last_active":"2023-11-28T11:51:19.093331+0000","last_peered":"2023-11-28T11:51:19.093331+0000","last_clean":"2023-11-28T11:51:19.093331+0000","last_became_active":"2023-11-28T11:51:16.885744+0000","last_became_peered":"2023-11-28T11:51:16.885744+0000","last_unstale":"2023-11-28T11:51:19.093331+0000","last_undegraded":"2023-11-28T11:51:19.093331+0000","last_fullsized":"2023-11-28T11:51:19.093331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:20:59.112675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879642+0000","last_change":"2023-11-28T11:51:16.898873+0000","last_active":"2023-11-28T11:51:17.879642+0000","last_peered":"2023-11-28T11:51:17.879642+0000","last_clean":"2023-11-28T11:51:17.879642+0000","last_became_active":"2023-11-28T11:51:16.898567+0000","last_became_peered":"2023-11-28T11:51:16.898567+0000","last_unstale":"2023-11-28T11:51:17.879642+0000","last_undegraded":"2023-11-28T11:51:17.879642+0000","last_fullsized":"2023-11-28T11:51:17.879642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881611+0000","last_change":"2023-11-28T11:51:16.892348+0000","last_active":"2023-11-28T11:51:17.881611+0000","last_peered":"2023-11-28T11:51:17.881611+0000","last_clean":"2023-11-28T11:51:17.881611+0000","last_became_active":"2023-11-28T11:51:16.891451+0000","last_became_peered":"2023-11-28T11:51:16.891451+0000","last_unstale":"2023-11-28T11:51:17.881611+0000","last_undegraded":"2023-11-28T11:51:17.881611+0000","last_fullsized":"2023-11-28T11:51:17.881611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:37.035204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092706+0000","last_change":"2023-11-28T11:51:16.893555+0000","last_active":"2023-11-28T11:51:19.092706+0000","last_peered":"2023-11-28T11:51:19.092706+0000","last_clean":"2023-11-28T11:51:19.092706+0000","last_became_active":"2023-11-28T11:51:16.893296+0000","last_became_peered":"2023-11-28T11:51:16.893296+0000","last_unstale":"2023-11-28T11:51:19.092706+0000","last_undegraded":"2023-11-28T11:51:19.092706+0000","last_fullsized":"2023-11-28T11:51:19.092706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:13.298585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092713+0000","last_change":"2023-11-28T11:51:16.885357+0000","last_active":"2023-11-28T11:51:19.092713+0000","last_peered":"2023-11-28T11:51:19.092713+0000","last_clean":"2023-11-28T11:51:19.092713+0000","last_became_active":"2023-11-28T11:51:16.885098+0000","last_became_peered":"2023-11-28T11:51:16.885098+0000","last_unstale":"2023-11-28T11:51:19.092713+0000","last_undegraded":"2023-11-28T11:51:19.092713+0000","last_fullsized":"2023-11-28T11:51:19.092713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:18.850790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880260+0000","last_change":"2023-11-28T11:51:16.882443+0000","last_active":"2023-11-28T11:51:17.880260+0000","last_peered":"2023-11-28T11:51:17.880260+0000","last_clean":"2023-11-28T11:51:17.880260+0000","last_became_active":"2023-11-28T11:51:16.882164+0000","last_became_peered":"2023-11-28T11:51:16.882164+0000","last_unstale":"2023-11-28T11:51:17.880260+0000","last_undegraded":"2023-11-28T11:51:17.880260+0000","last_fullsized":"2023-11-28T11:51:17.880260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093567+0000","last_change":"2023-11-28T11:51:16.895797+0000","last_active":"2023-11-28T11:51:19.093567+0000","last_peered":"2023-11-28T11:51:19.093567+0000","last_clean":"2023-11-28T11:51:19.093567+0000","last_became_active":"2023-11-28T11:51:16.895517+0000","last_became_peered":"2023-11-28T11:51:16.895517+0000","last_unstale":"2023-11-28T11:51:19.093567+0000","last_undegraded":"2023-11-28T11:51:19.093567+0000","last_fullsized":"2023-11-28T11:51:19.093567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881087+0000","last_change":"2023-11-28T11:51:16.899351+0000","last_active":"2023-11-28T11:51:17.881087+0000","last_peered":"2023-11-28T11:51:17.881087+0000","last_clean":"2023-11-28T11:51:17.881087+0000","last_became_active":"2023-11-28T11:51:16.899044+0000","last_became_peered":"2023-11-28T11:51:16.899044+0000","last_unstale":"2023-11-28T11:51:17.881087+0000","last_undegraded":"2023-11-28T11:51:17.881087+0000","last_fullsized":"2023-11-28T11:51:17.881087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:09.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093414+0000","last_change":"2023-11-28T11:51:16.885961+0000","last_active":"2023-11-28T11:51:19.093414+0000","last_peered":"2023-11-28T11:51:19.093414+0000","last_clean":"2023-11-28T11:51:19.093414+0000","last_became_active":"2023-11-28T11:51:16.885709+0000","last_became_peered":"2023-11-28T11:51:16.885709+0000","last_unstale":"2023-11-28T11:51:19.093414+0000","last_undegraded":"2023-11-28T11:51:19.093414+0000","last_fullsized":"2023-11-28T11:51:19.093414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:13.291091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881602+0000","last_change":"2023-11-28T11:51:16.893897+0000","last_active":"2023-11-28T11:51:17.881602+0000","last_peered":"2023-11-28T11:51:17.881602+0000","last_clean":"2023-11-28T11:51:17.881602+0000","last_became_active":"2023-11-28T11:51:16.893541+0000","last_became_peered":"2023-11-28T11:51:16.893541+0000","last_unstale":"2023-11-28T11:51:17.881602+0000","last_undegraded":"2023-11-28T11:51:17.881602+0000","last_fullsized":"2023-11-28T11:51:17.881602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092888+0000","last_change":"2023-11-28T11:51:16.887327+0000","last_active":"2023-11-28T11:51:19.092888+0000","last_peered":"2023-11-28T11:51:19.092888+0000","last_clean":"2023-11-28T11:51:19.092888+0000","last_became_active":"2023-11-28T11:51:16.887062+0000","last_became_peered":"2023-11-28T11:51:16.887062+0000","last_unstale":"2023-11-28T11:51:19.092888+0000","last_undegraded":"2023-11-28T11:51:19.092888+0000","last_fullsized":"2023-11-28T11:51:19.092888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:49.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881480+0000","last_change":"2023-11-28T11:51:16.889122+0000","last_active":"2023-11-28T11:51:17.881480+0000","last_peered":"2023-11-28T11:51:17.881480+0000","last_clean":"2023-11-28T11:51:17.881480+0000","last_became_active":"2023-11-28T11:51:16.888849+0000","last_became_peered":"2023-11-28T11:51:16.888849+0000","last_unstale":"2023-11-28T11:51:17.881480+0000","last_undegraded":"2023-11-28T11:51:17.881480+0000","last_fullsized":"2023-11-28T11:51:17.881480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093486+0000","last_change":"2023-11-28T11:51:16.895113+0000","last_active":"2023-11-28T11:51:19.093486+0000","last_peered":"2023-11-28T11:51:19.093486+0000","last_clean":"2023-11-28T11:51:19.093486+0000","last_became_active":"2023-11-28T11:51:16.894793+0000","last_became_peered":"2023-11-28T11:51:16.894793+0000","last_unstale":"2023-11-28T11:51:19.093486+0000","last_undegraded":"2023-11-28T11:51:19.093486+0000","last_fullsized":"2023-11-28T11:51:19.093486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898452+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897946+0000","last_became_peered":"2023-11-28T11:51:16.897946+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:20.222278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880183+0000","last_change":"2023-11-28T11:51:16.898903+0000","last_active":"2023-11-28T11:51:17.880183+0000","last_peered":"2023-11-28T11:51:17.880183+0000","last_clean":"2023-11-28T11:51:17.880183+0000","last_became_active":"2023-11-28T11:51:16.898719+0000","last_became_peered":"2023-11-28T11:51:16.898719+0000","last_unstale":"2023-11-28T11:51:17.880183+0000","last_undegraded":"2023-11-28T11:51:17.880183+0000","last_fullsized":"2023-11-28T11:51:17.880183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093156+0000","last_change":"2023-11-28T11:51:16.895329+0000","last_active":"2023-11-28T11:51:19.093156+0000","last_peered":"2023-11-28T11:51:19.093156+0000","last_clean":"2023-11-28T11:51:19.093156+0000","last_became_active":"2023-11-28T11:51:16.894998+0000","last_became_peered":"2023-11-28T11:51:16.894998+0000","last_unstale":"2023-11-28T11:51:19.093156+0000","last_undegraded":"2023-11-28T11:51:19.093156+0000","last_fullsized":"2023-11-28T11:51:19.093156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093690+0000","last_change":"2023-11-28T11:51:16.889063+0000","last_active":"2023-11-28T11:51:19.093690+0000","last_peered":"2023-11-28T11:51:19.093690+0000","last_clean":"2023-11-28T11:51:19.093690+0000","last_became_active":"2023-11-28T11:51:16.888711+0000","last_became_peered":"2023-11-28T11:51:16.888711+0000","last_unstale":"2023-11-28T11:51:19.093690+0000","last_undegraded":"2023-11-28T11:51:19.093690+0000","last_fullsized":"2023-11-28T11:51:19.093690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093545+0000","last_change":"2023-11-28T11:51:16.882290+0000","last_active":"2023-11-28T11:51:19.093545+0000","last_peered":"2023-11-28T11:51:19.093545+0000","last_clean":"2023-11-28T11:51:19.093545+0000","last_became_active":"2023-11-28T11:51:16.882049+0000","last_became_peered":"2023-11-28T11:51:16.882049+0000","last_unstale":"2023-11-28T11:51:19.093545+0000","last_undegraded":"2023-11-28T11:51:19.093545+0000","last_fullsized":"2023-11-28T11:51:19.093545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879435+0000","last_change":"2023-11-28T11:51:16.896720+0000","last_active":"2023-11-28T11:51:17.879435+0000","last_peered":"2023-11-28T11:51:17.879435+0000","last_clean":"2023-11-28T11:51:17.879435+0000","last_became_active":"2023-11-28T11:51:16.896429+0000","last_became_peered":"2023-11-28T11:51:16.896429+0000","last_unstale":"2023-11-28T11:51:17.879435+0000","last_undegraded":"2023-11-28T11:51:17.879435+0000","last_fullsized":"2023-11-28T11:51:17.879435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881315+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:17.881315+0000","last_peered":"2023-11-28T11:51:17.881315+0000","last_clean":"2023-11-28T11:51:17.881315+0000","last_became_active":"2023-11-28T11:51:16.482973+0000","last_became_peered":"2023-11-28T11:51:16.482973+0000","last_unstale":"2023-11-28T11:51:17.881315+0000","last_undegraded":"2023-11-28T11:51:17.881315+0000","last_fullsized":"2023-11-28T11:51:17.881315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880643+0000","last_change":"2023-11-28T11:51:16.887525+0000","last_active":"2023-11-28T11:51:17.880643+0000","last_peered":"2023-11-28T11:51:17.880643+0000","last_clean":"2023-11-28T11:51:17.880643+0000","last_became_active":"2023-11-28T11:51:16.887247+0000","last_became_peered":"2023-11-28T11:51:16.887247+0000","last_unstale":"2023-11-28T11:51:17.880643+0000","last_undegraded":"2023-11-28T11:51:17.880643+0000","last_fullsized":"2023-11-28T11:51:17.880643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093032+0000","last_change":"2023-11-28T11:51:16.481846+0000","last_active":"2023-11-28T11:51:19.093032+0000","last_peered":"2023-11-28T11:51:19.093032+0000","last_clean":"2023-11-28T11:51:19.093032+0000","last_became_active":"2023-11-28T11:51:16.481653+0000","last_became_peered":"2023-11-28T11:51:16.481653+0000","last_unstale":"2023-11-28T11:51:19.093032+0000","last_undegraded":"2023-11-28T11:51:19.093032+0000","last_fullsized":"2023-11-28T11:51:19.093032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:21.555920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093520+0000","last_change":"2023-11-28T11:51:16.895323+0000","last_active":"2023-11-28T11:51:19.093520+0000","last_peered":"2023-11-28T11:51:19.093520+0000","last_clean":"2023-11-28T11:51:19.093520+0000","last_became_active":"2023-11-28T11:51:16.895030+0000","last_became_peered":"2023-11-28T11:51:16.895030+0000","last_unstale":"2023-11-28T11:51:19.093520+0000","last_undegraded":"2023-11-28T11:51:19.093520+0000","last_fullsized":"2023-11-28T11:51:19.093520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:59.340661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880302+0000","last_change":"2023-11-28T11:51:16.488974+0000","last_active":"2023-11-28T11:51:17.880302+0000","last_peered":"2023-11-28T11:51:17.880302+0000","last_clean":"2023-11-28T11:51:17.880302+0000","last_became_active":"2023-11-28T11:51:16.488697+0000","last_became_peered":"2023-11-28T11:51:16.488697+0000","last_unstale":"2023-11-28T11:51:17.880302+0000","last_undegraded":"2023-11-28T11:51:17.880302+0000","last_fullsized":"2023-11-28T11:51:17.880302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093841+0000","last_change":"2023-11-28T11:51:16.888129+0000","last_active":"2023-11-28T11:51:19.093841+0000","last_peered":"2023-11-28T11:51:19.093841+0000","last_clean":"2023-11-28T11:51:19.093841+0000","last_became_active":"2023-11-28T11:51:16.887848+0000","last_became_peered":"2023-11-28T11:51:16.887848+0000","last_unstale":"2023-11-28T11:51:19.093841+0000","last_undegraded":"2023-11-28T11:51:19.093841+0000","last_fullsized":"2023-11-28T11:51:19.093841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:42.239688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093488+0000","last_change":"2023-11-28T11:51:16.488283+0000","last_active":"2023-11-28T11:51:19.093488+0000","last_peered":"2023-11-28T11:51:19.093488+0000","last_clean":"2023-11-28T11:51:19.093488+0000","last_became_active":"2023-11-28T11:51:16.487994+0000","last_became_peered":"2023-11-28T11:51:16.487994+0000","last_unstale":"2023-11-28T11:51:19.093488+0000","last_undegraded":"2023-11-28T11:51:19.093488+0000","last_fullsized":"2023-11-28T11:51:19.093488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:04.545703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880313+0000","last_change":"2023-11-28T11:51:16.891442+0000","last_active":"2023-11-28T11:51:17.880313+0000","last_peered":"2023-11-28T11:51:17.880313+0000","last_clean":"2023-11-28T11:51:17.880313+0000","last_became_active":"2023-11-28T11:51:16.891180+0000","last_became_peered":"2023-11-28T11:51:16.891180+0000","last_unstale":"2023-11-28T11:51:17.880313+0000","last_undegraded":"2023-11-28T11:51:17.880313+0000","last_fullsized":"2023-11-28T11:51:17.880313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879563+0000","last_change":"2023-11-28T11:51:16.484401+0000","last_active":"2023-11-28T11:51:17.879563+0000","last_peered":"2023-11-28T11:51:17.879563+0000","last_clean":"2023-11-28T11:51:17.879563+0000","last_became_active":"2023-11-28T11:51:16.484103+0000","last_became_peered":"2023-11-28T11:51:16.484103+0000","last_unstale":"2023-11-28T11:51:17.879563+0000","last_undegraded":"2023-11-28T11:51:17.879563+0000","last_fullsized":"2023-11-28T11:51:17.879563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880312+0000","last_change":"2023-11-28T11:51:16.898526+0000","last_active":"2023-11-28T11:51:17.880312+0000","last_peered":"2023-11-28T11:51:17.880312+0000","last_clean":"2023-11-28T11:51:17.880312+0000","last_became_active":"2023-11-28T11:51:16.897658+0000","last_became_peered":"2023-11-28T11:51:16.897658+0000","last_unstale":"2023-11-28T11:51:17.880312+0000","last_undegraded":"2023-11-28T11:51:17.880312+0000","last_fullsized":"2023-11-28T11:51:17.880312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093666+0000","last_change":"2023-11-28T11:51:16.484674+0000","last_active":"2023-11-28T11:51:19.093666+0000","last_peered":"2023-11-28T11:51:19.093666+0000","last_clean":"2023-11-28T11:51:19.093666+0000","last_became_active":"2023-11-28T11:51:16.484350+0000","last_became_peered":"2023-11-28T11:51:16.484350+0000","last_unstale":"2023-11-28T11:51:19.093666+0000","last_undegraded":"2023-11-28T11:51:19.093666+0000","last_fullsized":"2023-11-28T11:51:19.093666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:13.351037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093718+0000","last_change":"2023-11-28T11:51:16.889523+0000","last_active":"2023-11-28T11:51:19.093718+0000","last_peered":"2023-11-28T11:51:19.093718+0000","last_clean":"2023-11-28T11:51:19.093718+0000","last_became_active":"2023-11-28T11:51:16.889323+0000","last_became_peered":"2023-11-28T11:51:16.889323+0000","last_unstale":"2023-11-28T11:51:19.093718+0000","last_undegraded":"2023-11-28T11:51:19.093718+0000","last_fullsized":"2023-11-28T11:51:19.093718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881658+0000","last_change":"2023-11-28T11:51:16.482345+0000","last_active":"2023-11-28T11:51:17.881658+0000","last_peered":"2023-11-28T11:51:17.881658+0000","last_clean":"2023-11-28T11:51:17.881658+0000","last_became_active":"2023-11-28T11:51:16.482106+0000","last_became_peered":"2023-11-28T11:51:16.482106+0000","last_unstale":"2023-11-28T11:51:17.881658+0000","last_undegraded":"2023-11-28T11:51:17.881658+0000","last_fullsized":"2023-11-28T11:51:17.881658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881274+0000","last_change":"2023-11-28T11:51:16.890129+0000","last_active":"2023-11-28T11:51:17.881274+0000","last_peered":"2023-11-28T11:51:17.881274+0000","last_clean":"2023-11-28T11:51:17.881274+0000","last_became_active":"2023-11-28T11:51:16.889849+0000","last_became_peered":"2023-11-28T11:51:16.889849+0000","last_unstale":"2023-11-28T11:51:17.881274+0000","last_undegraded":"2023-11-28T11:51:17.881274+0000","last_fullsized":"2023-11-28T11:51:17.881274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093375+0000","last_change":"2023-11-28T11:51:16.485194+0000","last_active":"2023-11-28T11:51:19.093375+0000","last_peered":"2023-11-28T11:51:19.093375+0000","last_clean":"2023-11-28T11:51:19.093375+0000","last_became_active":"2023-11-28T11:51:16.484873+0000","last_became_peered":"2023-11-28T11:51:16.484873+0000","last_unstale":"2023-11-28T11:51:19.093375+0000","last_undegraded":"2023-11-28T11:51:19.093375+0000","last_fullsized":"2023-11-28T11:51:19.093375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880521+0000","last_change":"2023-11-28T11:51:16.887321+0000","last_active":"2023-11-28T11:51:17.880521+0000","last_peered":"2023-11-28T11:51:17.880521+0000","last_clean":"2023-11-28T11:51:17.880521+0000","last_became_active":"2023-11-28T11:51:16.886989+0000","last_became_peered":"2023-11-28T11:51:16.886989+0000","last_unstale":"2023-11-28T11:51:17.880521+0000","last_undegraded":"2023-11-28T11:51:17.880521+0000","last_fullsized":"2023-11-28T11:51:17.880521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093431+0000","last_change":"2023-11-28T11:51:16.486311+0000","last_active":"2023-11-28T11:51:19.093431+0000","last_peered":"2023-11-28T11:51:19.093431+0000","last_clean":"2023-11-28T11:51:19.093431+0000","last_became_active":"2023-11-28T11:51:16.485952+0000","last_became_peered":"2023-11-28T11:51:16.485952+0000","last_unstale":"2023-11-28T11:51:19.093431+0000","last_undegraded":"2023-11-28T11:51:19.093431+0000","last_fullsized":"2023-11-28T11:51:19.093431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:13.615057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879568+0000","last_change":"2023-11-28T11:51:16.896713+0000","last_active":"2023-11-28T11:51:17.879568+0000","last_peered":"2023-11-28T11:51:17.879568+0000","last_clean":"2023-11-28T11:51:17.879568+0000","last_became_active":"2023-11-28T11:51:16.896490+0000","last_became_peered":"2023-11-28T11:51:16.896490+0000","last_unstale":"2023-11-28T11:51:17.879568+0000","last_undegraded":"2023-11-28T11:51:17.879568+0000","last_fullsized":"2023-11-28T11:51:17.879568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093447+0000","last_change":"2023-11-28T11:51:16.485718+0000","last_active":"2023-11-28T11:51:19.093447+0000","last_peered":"2023-11-28T11:51:19.093447+0000","last_clean":"2023-11-28T11:51:19.093447+0000","last_became_active":"2023-11-28T11:51:16.485279+0000","last_became_peered":"2023-11-28T11:51:16.485279+0000","last_unstale":"2023-11-28T11:51:19.093447+0000","last_undegraded":"2023-11-28T11:51:19.093447+0000","last_fullsized":"2023-11-28T11:51:19.093447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898233+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897971+0000","last_became_peered":"2023-11-28T11:51:16.897971+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:59.713494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092990+0000","last_change":"2023-11-28T11:51:16.489032+0000","last_active":"2023-11-28T11:51:19.092990+0000","last_peered":"2023-11-28T11:51:19.092990+0000","last_clean":"2023-11-28T11:51:19.092990+0000","last_became_active":"2023-11-28T11:51:16.488752+0000","last_became_peered":"2023-11-28T11:51:16.488752+0000","last_unstale":"2023-11-28T11:51:19.092990+0000","last_undegraded":"2023-11-28T11:51:19.092990+0000","last_fullsized":"2023-11-28T11:51:19.092990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880771+0000","last_change":"2023-11-28T11:51:16.897679+0000","last_active":"2023-11-28T11:51:17.880771+0000","last_peered":"2023-11-28T11:51:17.880771+0000","last_clean":"2023-11-28T11:51:17.880771+0000","last_became_active":"2023-11-28T11:51:16.897336+0000","last_became_peered":"2023-11-28T11:51:16.897336+0000","last_unstale":"2023-11-28T11:51:17.880771+0000","last_undegraded":"2023-11-28T11:51:17.880771+0000","last_fullsized":"2023-11-28T11:51:17.880771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093751+0000","last_change":"2023-11-28T11:51:16.485545+0000","last_active":"2023-11-28T11:51:19.093751+0000","last_peered":"2023-11-28T11:51:19.093751+0000","last_clean":"2023-11-28T11:51:19.093751+0000","last_became_active":"2023-11-28T11:51:16.485097+0000","last_became_peered":"2023-11-28T11:51:16.485097+0000","last_unstale":"2023-11-28T11:51:19.093751+0000","last_undegraded":"2023-11-28T11:51:19.093751+0000","last_fullsized":"2023-11-28T11:51:19.093751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:41.232205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.896139+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.895569+0000","last_became_peered":"2023-11-28T11:51:16.895569+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093685+0000","last_change":"2023-11-28T11:51:16.485744+0000","last_active":"2023-11-28T11:51:19.093685+0000","last_peered":"2023-11-28T11:51:19.093685+0000","last_clean":"2023-11-28T11:51:19.093685+0000","last_became_active":"2023-11-28T11:51:16.485468+0000","last_became_peered":"2023-11-28T11:51:16.485468+0000","last_unstale":"2023-11-28T11:51:19.093685+0000","last_undegraded":"2023-11-28T11:51:19.093685+0000","last_fullsized":"2023-11-28T11:51:19.093685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881144+0000","last_change":"2023-11-28T11:51:16.887450+0000","last_active":"2023-11-28T11:51:17.881144+0000","last_peered":"2023-11-28T11:51:17.881144+0000","last_clean":"2023-11-28T11:51:17.881144+0000","last_became_active":"2023-11-28T11:51:16.887190+0000","last_became_peered":"2023-11-28T11:51:16.887190+0000","last_unstale":"2023-11-28T11:51:17.881144+0000","last_undegraded":"2023-11-28T11:51:17.881144+0000","last_fullsized":"2023-11-28T11:51:17.881144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:00.113811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093607+0000","last_change":"2023-11-28T11:51:16.485512+0000","last_active":"2023-11-28T11:51:19.093607+0000","last_peered":"2023-11-28T11:51:19.093607+0000","last_clean":"2023-11-28T11:51:19.093607+0000","last_became_active":"2023-11-28T11:51:16.485186+0000","last_became_peered":"2023-11-28T11:51:16.485186+0000","last_unstale":"2023-11-28T11:51:19.093607+0000","last_undegraded":"2023-11-28T11:51:19.093607+0000","last_fullsized":"2023-11-28T11:51:19.093607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:41.955746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093236+0000","last_change":"2023-11-28T11:51:16.888771+0000","last_active":"2023-11-28T11:51:19.093236+0000","last_peered":"2023-11-28T11:51:19.093236+0000","last_clean":"2023-11-28T11:51:19.093236+0000","last_became_active":"2023-11-28T11:51:16.888179+0000","last_became_peered":"2023-11-28T11:51:16.888179+0000","last_unstale":"2023-11-28T11:51:19.093236+0000","last_undegraded":"2023-11-28T11:51:19.093236+0000","last_fullsized":"2023-11-28T11:51:19.093236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879495+0000","last_change":"2023-11-28T11:51:16.475494+0000","last_active":"2023-11-28T11:51:17.879495+0000","last_peered":"2023-11-28T11:51:17.879495+0000","last_clean":"2023-11-28T11:51:17.879495+0000","last_became_active":"2023-11-28T11:51:16.475189+0000","last_became_peered":"2023-11-28T11:51:16.475189+0000","last_unstale":"2023-11-28T11:51:17.879495+0000","last_undegraded":"2023-11-28T11:51:17.879495+0000","last_fullsized":"2023-11-28T11:51:17.879495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092876+0000","last_change":"2023-11-28T11:51:16.889569+0000","last_active":"2023-11-28T11:51:19.092876+0000","last_peered":"2023-11-28T11:51:19.092876+0000","last_clean":"2023-11-28T11:51:19.092876+0000","last_became_active":"2023-11-28T11:51:16.888349+0000","last_became_peered":"2023-11-28T11:51:16.888349+0000","last_unstale":"2023-11-28T11:51:19.092876+0000","last_undegraded":"2023-11-28T11:51:19.092876+0000","last_fullsized":"2023-11-28T11:51:19.092876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:37.975550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880421+0000","last_change":"2023-11-28T11:51:16.491156+0000","last_active":"2023-11-28T11:51:17.880421+0000","last_peered":"2023-11-28T11:51:17.880421+0000","last_clean":"2023-11-28T11:51:17.880421+0000","last_became_active":"2023-11-28T11:51:16.490844+0000","last_became_peered":"2023-11-28T11:51:16.490844+0000","last_unstale":"2023-11-28T11:51:17.880421+0000","last_undegraded":"2023-11-28T11:51:17.880421+0000","last_fullsized":"2023-11-28T11:51:17.880421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881598+0000","last_change":"2023-11-28T11:51:16.890271+0000","last_active":"2023-11-28T11:51:17.881598+0000","last_peered":"2023-11-28T11:51:17.881598+0000","last_clean":"2023-11-28T11:51:17.881598+0000","last_became_active":"2023-11-28T11:51:16.889985+0000","last_became_peered":"2023-11-28T11:51:16.889985+0000","last_unstale":"2023-11-28T11:51:17.881598+0000","last_undegraded":"2023-11-28T11:51:17.881598+0000","last_fullsized":"2023-11-28T11:51:17.881598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:03.275473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093142+0000","last_change":"2023-11-28T11:51:16.482742+0000","last_active":"2023-11-28T11:51:19.093142+0000","last_peered":"2023-11-28T11:51:19.093142+0000","last_clean":"2023-11-28T11:51:19.093142+0000","last_became_active":"2023-11-28T11:51:16.482411+0000","last_became_peered":"2023-11-28T11:51:16.482411+0000","last_unstale":"2023-11-28T11:51:19.093142+0000","last_undegraded":"2023-11-28T11:51:19.093142+0000","last_fullsized":"2023-11-28T11:51:19.093142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881716+0000","last_change":"2023-11-28T11:51:16.893619+0000","last_active":"2023-11-28T11:51:17.881716+0000","last_peered":"2023-11-28T11:51:17.881716+0000","last_clean":"2023-11-28T11:51:17.881716+0000","last_became_active":"2023-11-28T11:51:16.893353+0000","last_became_peered":"2023-11-28T11:51:16.893353+0000","last_unstale":"2023-11-28T11:51:17.881716+0000","last_undegraded":"2023-11-28T11:51:17.881716+0000","last_fullsized":"2023-11-28T11:51:17.881716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881245+0000","last_change":"2023-11-28T11:51:16.480268+0000","last_active":"2023-11-28T11:51:17.881245+0000","last_peered":"2023-11-28T11:51:17.881245+0000","last_clean":"2023-11-28T11:51:17.881245+0000","last_became_active":"2023-11-28T11:51:16.479377+0000","last_became_peered":"2023-11-28T11:51:16.479377+0000","last_unstale":"2023-11-28T11:51:17.881245+0000","last_undegraded":"2023-11-28T11:51:17.881245+0000","last_fullsized":"2023-11-28T11:51:17.881245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881631+0000","last_change":"2023-11-28T11:51:16.890147+0000","last_active":"2023-11-28T11:51:17.881631+0000","last_peered":"2023-11-28T11:51:17.881631+0000","last_clean":"2023-11-28T11:51:17.881631+0000","last_became_active":"2023-11-28T11:51:16.889833+0000","last_became_peered":"2023-11-28T11:51:16.889833+0000","last_unstale":"2023-11-28T11:51:17.881631+0000","last_undegraded":"2023-11-28T11:51:17.881631+0000","last_fullsized":"2023-11-28T11:51:17.881631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880209+0000","last_change":"2023-11-28T11:51:16.478507+0000","last_active":"2023-11-28T11:51:17.880209+0000","last_peered":"2023-11-28T11:51:17.880209+0000","last_clean":"2023-11-28T11:51:17.880209+0000","last_became_active":"2023-11-28T11:51:16.478034+0000","last_became_peered":"2023-11-28T11:51:16.478034+0000","last_unstale":"2023-11-28T11:51:17.880209+0000","last_undegraded":"2023-11-28T11:51:17.880209+0000","last_fullsized":"2023-11-28T11:51:17.880209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093877+0000","last_change":"2023-11-28T11:51:16.885456+0000","last_active":"2023-11-28T11:51:19.093877+0000","last_peered":"2023-11-28T11:51:19.093877+0000","last_clean":"2023-11-28T11:51:19.093877+0000","last_became_active":"2023-11-28T11:51:16.885208+0000","last_became_peered":"2023-11-28T11:51:16.885208+0000","last_unstale":"2023-11-28T11:51:19.093877+0000","last_undegraded":"2023-11-28T11:51:19.093877+0000","last_fullsized":"2023-11-28T11:51:19.093877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:24.014435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880712+0000","last_change":"2023-11-28T11:51:16.483690+0000","last_active":"2023-11-28T11:51:17.880712+0000","last_peered":"2023-11-28T11:51:17.880712+0000","last_clean":"2023-11-28T11:51:17.880712+0000","last_became_active":"2023-11-28T11:51:16.483351+0000","last_became_peered":"2023-11-28T11:51:16.483351+0000","last_unstale":"2023-11-28T11:51:17.880712+0000","last_undegraded":"2023-11-28T11:51:17.880712+0000","last_fullsized":"2023-11-28T11:51:17.880712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877898+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:51:16.877898+0000","last_peered":"2023-11-28T11:51:16.877898+0000","last_clean":"2023-11-28T11:51:16.877898+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:51:16.877898+0000","last_undegraded":"2023-11-28T11:51:16.877898+0000","last_fullsized":"2023-11-28T11:51:16.877898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880121+0000","last_change":"2023-11-28T11:51:16.897901+0000","last_active":"2023-11-28T11:51:17.880121+0000","last_peered":"2023-11-28T11:51:17.880121+0000","last_clean":"2023-11-28T11:51:17.880121+0000","last_became_active":"2023-11-28T11:51:16.897621+0000","last_became_peered":"2023-11-28T11:51:16.897621+0000","last_unstale":"2023-11-28T11:51:17.880121+0000","last_undegraded":"2023-11-28T11:51:17.880121+0000","last_fullsized":"2023-11-28T11:51:17.880121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876559+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:51:16.876559+0000","last_peered":"2023-11-28T11:51:16.876559+0000","last_clean":"2023-11-28T11:51:16.876559+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:51:16.876559+0000","last_undegraded":"2023-11-28T11:51:16.876559+0000","last_fullsized":"2023-11-28T11:51:16.876559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:20.007269+0000","last_change":"2023-11-28T11:51:16.486571+0000","last_active":"2023-11-28T11:51:20.007269+0000","last_peered":"2023-11-28T11:51:20.007269+0000","last_clean":"2023-11-28T11:51:20.007269+0000","last_became_active":"2023-11-28T11:51:16.486338+0000","last_became_peered":"2023-11-28T11:51:16.486338+0000","last_unstale":"2023-11-28T11:51:20.007269+0000","last_undegraded":"2023-11-28T11:51:20.007269+0000","last_fullsized":"2023-11-28T11:51:20.007269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:54.333661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881136+0000","last_change":"2023-11-28T11:51:16.889991+0000","last_active":"2023-11-28T11:51:17.881136+0000","last_peered":"2023-11-28T11:51:17.881136+0000","last_clean":"2023-11-28T11:51:17.881136+0000","last_became_active":"2023-11-28T11:51:16.889694+0000","last_became_peered":"2023-11-28T11:51:16.889694+0000","last_unstale":"2023-11-28T11:51:17.881136+0000","last_undegraded":"2023-11-28T11:51:17.881136+0000","last_fullsized":"2023-11-28T11:51:17.881136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.870699+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:51:16.870699+0000","last_peered":"2023-11-28T11:51:16.870699+0000","last_clean":"2023-11-28T11:51:16.870699+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:51:16.870699+0000","last_undegraded":"2023-11-28T11:51:16.870699+0000","last_fullsized":"2023-11-28T11:51:16.870699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877332+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:51:16.877332+0000","last_peered":"2023-11-28T11:51:16.877332+0000","last_clean":"2023-11-28T11:51:16.877332+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:51:16.877332+0000","last_undegraded":"2023-11-28T11:51:16.877332+0000","last_fullsized":"2023-11-28T11:51:16.877332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093112+0000","last_change":"2023-11-28T11:51:16.486764+0000","last_active":"2023-11-28T11:51:19.093112+0000","last_peered":"2023-11-28T11:51:19.093112+0000","last_clean":"2023-11-28T11:51:19.093112+0000","last_became_active":"2023-11-28T11:51:16.486448+0000","last_became_peered":"2023-11-28T11:51:16.486448+0000","last_unstale":"2023-11-28T11:51:19.093112+0000","last_undegraded":"2023-11-28T11:51:19.093112+0000","last_fullsized":"2023-11-28T11:51:19.093112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:15.029206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093385+0000","last_change":"2023-11-28T11:51:16.888008+0000","last_active":"2023-11-28T11:51:19.093385+0000","last_peered":"2023-11-28T11:51:19.093385+0000","last_clean":"2023-11-28T11:51:19.093385+0000","last_became_active":"2023-11-28T11:51:16.887745+0000","last_became_peered":"2023-11-28T11:51:16.887745+0000","last_unstale":"2023-11-28T11:51:19.093385+0000","last_undegraded":"2023-11-28T11:51:19.093385+0000","last_fullsized":"2023-11-28T11:51:19.093385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:39.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876792+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:51:16.876792+0000","last_peered":"2023-11-28T11:51:16.876792+0000","last_clean":"2023-11-28T11:51:16.876792+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:51:16.876792+0000","last_undegraded":"2023-11-28T11:51:16.876792+0000","last_fullsized":"2023-11-28T11:51:16.876792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092461+0000","last_change":"2023-11-28T11:51:16.484243+0000","last_active":"2023-11-28T11:51:19.092461+0000","last_peered":"2023-11-28T11:51:19.092461+0000","last_clean":"2023-11-28T11:51:19.092461+0000","last_became_active":"2023-11-28T11:51:16.482924+0000","last_became_peered":"2023-11-28T11:51:16.482924+0000","last_unstale":"2023-11-28T11:51:19.092461+0000","last_undegraded":"2023-11-28T11:51:19.092461+0000","last_fullsized":"2023-11-28T11:51:19.092461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093787+0000","last_change":"2023-11-28T11:51:16.878987+0000","last_active":"2023-11-28T11:51:19.093787+0000","last_peered":"2023-11-28T11:51:19.093787+0000","last_clean":"2023-11-28T11:51:19.093787+0000","last_became_active":"2023-11-28T11:51:16.878798+0000","last_became_peered":"2023-11-28T11:51:16.878798+0000","last_unstale":"2023-11-28T11:51:19.093787+0000","last_undegraded":"2023-11-28T11:51:19.093787+0000","last_fullsized":"2023-11-28T11:51:19.093787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873448+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:51:16.873448+0000","last_peered":"2023-11-28T11:51:16.873448+0000","last_clean":"2023-11-28T11:51:16.873448+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:51:16.873448+0000","last_undegraded":"2023-11-28T11:51:16.873448+0000","last_fullsized":"2023-11-28T11:51:16.873448+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880298+0000","last_change":"2023-11-28T11:51:16.476786+0000","last_active":"2023-11-28T11:51:17.880298+0000","last_peered":"2023-11-28T11:51:17.880298+0000","last_clean":"2023-11-28T11:51:17.880298+0000","last_became_active":"2023-11-28T11:51:16.476460+0000","last_became_peered":"2023-11-28T11:51:16.476460+0000","last_unstale":"2023-11-28T11:51:17.880298+0000","last_undegraded":"2023-11-28T11:51:17.880298+0000","last_fullsized":"2023-11-28T11:51:17.880298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.094045+0000","last_change":"2023-11-28T11:51:16.890298+0000","last_active":"2023-11-28T11:51:19.094045+0000","last_peered":"2023-11-28T11:51:19.094045+0000","last_clean":"2023-11-28T11:51:19.094045+0000","last_became_active":"2023-11-28T11:51:16.890029+0000","last_became_peered":"2023-11-28T11:51:16.890029+0000","last_unstale":"2023-11-28T11:51:19.094045+0000","last_undegraded":"2023-11-28T11:51:19.094045+0000","last_fullsized":"2023-11-28T11:51:19.094045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:42.963230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873404+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:51:16.873404+0000","last_peered":"2023-11-28T11:51:16.873404+0000","last_clean":"2023-11-28T11:51:16.873404+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:51:16.873404+0000","last_undegraded":"2023-11-28T11:51:16.873404+0000","last_fullsized":"2023-11-28T11:51:16.873404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881318+0000","last_change":"2023-11-28T11:51:16.488913+0000","last_active":"2023-11-28T11:51:17.881318+0000","last_peered":"2023-11-28T11:51:17.881318+0000","last_clean":"2023-11-28T11:51:17.881318+0000","last_became_active":"2023-11-28T11:51:16.488559+0000","last_became_peered":"2023-11-28T11:51:16.488559+0000","last_unstale":"2023-11-28T11:51:17.881318+0000","last_undegraded":"2023-11-28T11:51:17.881318+0000","last_fullsized":"2023-11-28T11:51:17.881318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093581+0000","last_change":"2023-11-28T11:51:16.882735+0000","last_active":"2023-11-28T11:51:19.093581+0000","last_peered":"2023-11-28T11:51:19.093581+0000","last_clean":"2023-11-28T11:51:19.093581+0000","last_became_active":"2023-11-28T11:51:16.882478+0000","last_became_peered":"2023-11-28T11:51:16.882478+0000","last_unstale":"2023-11-28T11:51:19.093581+0000","last_undegraded":"2023-11-28T11:51:19.093581+0000","last_fullsized":"2023-11-28T11:51:19.093581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882570+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:51:16.882570+0000","last_peered":"2023-11-28T11:51:16.882570+0000","last_clean":"2023-11-28T11:51:16.882570+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:51:16.882570+0000","last_undegraded":"2023-11-28T11:51:16.882570+0000","last_fullsized":"2023-11-28T11:51:16.882570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093667+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:19.093667+0000","last_peered":"2023-11-28T11:51:19.093667+0000","last_clean":"2023-11-28T11:51:19.093667+0000","last_became_active":"2023-11-28T11:51:16.482544+0000","last_became_peered":"2023-11-28T11:51:16.482544+0000","last_unstale":"2023-11-28T11:51:19.093667+0000","last_undegraded":"2023-11-28T11:51:19.093667+0000","last_fullsized":"2023-11-28T11:51:19.093667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:40.095304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093117+0000","last_change":"2023-11-28T11:51:16.888841+0000","last_active":"2023-11-28T11:51:19.093117+0000","last_peered":"2023-11-28T11:51:19.093117+0000","last_clean":"2023-11-28T11:51:19.093117+0000","last_became_active":"2023-11-28T11:51:16.886560+0000","last_became_peered":"2023-11-28T11:51:16.886560+0000","last_unstale":"2023-11-28T11:51:19.093117+0000","last_undegraded":"2023-11-28T11:51:19.093117+0000","last_fullsized":"2023-11-28T11:51:19.093117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:15.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882346+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:51:16.882346+0000","last_peered":"2023-11-28T11:51:16.882346+0000","last_clean":"2023-11-28T11:51:16.882346+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:51:16.882346+0000","last_undegraded":"2023-11-28T11:51:16.882346+0000","last_fullsized":"2023-11-28T11:51:16.882346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879570+0000","last_change":"2023-11-28T11:51:16.475455+0000","last_active":"2023-11-28T11:51:17.879570+0000","last_peered":"2023-11-28T11:51:17.879570+0000","last_clean":"2023-11-28T11:51:17.879570+0000","last_became_active":"2023-11-28T11:51:16.475159+0000","last_became_peered":"2023-11-28T11:51:16.475159+0000","last_unstale":"2023-11-28T11:51:17.879570+0000","last_undegraded":"2023-11-28T11:51:17.879570+0000","last_fullsized":"2023-11-28T11:51:17.879570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093769+0000","last_change":"2023-11-28T11:51:16.880638+0000","last_active":"2023-11-28T11:51:19.093769+0000","last_peered":"2023-11-28T11:51:19.093769+0000","last_clean":"2023-11-28T11:51:19.093769+0000","last_became_active":"2023-11-28T11:51:16.880329+0000","last_became_peered":"2023-11-28T11:51:16.880329+0000","last_unstale":"2023-11-28T11:51:19.093769+0000","last_undegraded":"2023-11-28T11:51:19.093769+0000","last_fullsized":"2023-11-28T11:51:19.093769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092710+0000","last_change":"2023-11-28T11:51:16.490914+0000","last_active":"2023-11-28T11:51:19.092710+0000","last_peered":"2023-11-28T11:51:19.092710+0000","last_clean":"2023-11-28T11:51:19.092710+0000","last_became_active":"2023-11-28T11:51:16.490638+0000","last_became_peered":"2023-11-28T11:51:16.490638+0000","last_unstale":"2023-11-28T11:51:19.092710+0000","last_undegraded":"2023-11-28T11:51:19.092710+0000","last_fullsized":"2023-11-28T11:51:19.092710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:42.375209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092793+0000","last_change":"2023-11-28T11:51:16.891450+0000","last_active":"2023-11-28T11:51:19.092793+0000","last_peered":"2023-11-28T11:51:19.092793+0000","last_clean":"2023-11-28T11:51:19.092793+0000","last_became_active":"2023-11-28T11:51:16.891183+0000","last_became_peered":"2023-11-28T11:51:16.891183+0000","last_unstale":"2023-11-28T11:51:19.092793+0000","last_undegraded":"2023-11-28T11:51:19.092793+0000","last_fullsized":"2023-11-28T11:51:19.092793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:44.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879331+0000","last_change":"2023-11-28T11:51:16.478457+0000","last_active":"2023-11-28T11:51:17.879331+0000","last_peered":"2023-11-28T11:51:17.879331+0000","last_clean":"2023-11-28T11:51:17.879331+0000","last_became_active":"2023-11-28T11:51:16.478170+0000","last_became_peered":"2023-11-28T11:51:16.478170+0000","last_unstale":"2023-11-28T11:51:17.879331+0000","last_undegraded":"2023-11-28T11:51:17.879331+0000","last_fullsized":"2023-11-28T11:51:17.879331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881448+0000","last_change":"2023-11-28T11:51:16.885759+0000","last_active":"2023-11-28T11:51:17.881448+0000","last_peered":"2023-11-28T11:51:17.881448+0000","last_clean":"2023-11-28T11:51:17.881448+0000","last_became_active":"2023-11-28T11:51:16.885461+0000","last_became_peered":"2023-11-28T11:51:16.885461+0000","last_unstale":"2023-11-28T11:51:17.881448+0000","last_undegraded":"2023-11-28T11:51:17.881448+0000","last_fullsized":"2023-11-28T11:51:17.881448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879652+0000","last_change":"2023-11-28T11:51:16.478962+0000","last_active":"2023-11-28T11:51:17.879652+0000","last_peered":"2023-11-28T11:51:17.879652+0000","last_clean":"2023-11-28T11:51:17.879652+0000","last_became_active":"2023-11-28T11:51:16.478687+0000","last_became_peered":"2023-11-28T11:51:16.478687+0000","last_unstale":"2023-11-28T11:51:17.879652+0000","last_undegraded":"2023-11-28T11:51:17.879652+0000","last_fullsized":"2023-11-28T11:51:17.879652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881454+0000","last_change":"2023-11-28T11:51:16.891408+0000","last_active":"2023-11-28T11:51:17.881454+0000","last_peered":"2023-11-28T11:51:17.881454+0000","last_clean":"2023-11-28T11:51:17.881454+0000","last_became_active":"2023-11-28T11:51:16.891119+0000","last_became_peered":"2023-11-28T11:51:16.891119+0000","last_unstale":"2023-11-28T11:51:17.881454+0000","last_undegraded":"2023-11-28T11:51:17.881454+0000","last_fullsized":"2023-11-28T11:51:17.881454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880602+0000","last_change":"2023-11-28T11:51:16.483925+0000","last_active":"2023-11-28T11:51:17.880602+0000","last_peered":"2023-11-28T11:51:17.880602+0000","last_clean":"2023-11-28T11:51:17.880602+0000","last_became_active":"2023-11-28T11:51:16.483640+0000","last_became_peered":"2023-11-28T11:51:16.483640+0000","last_unstale":"2023-11-28T11:51:17.880602+0000","last_undegraded":"2023-11-28T11:51:17.880602+0000","last_fullsized":"2023-11-28T11:51:17.880602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880383+0000","last_change":"2023-11-28T11:51:16.884604+0000","last_active":"2023-11-28T11:51:17.880383+0000","last_peered":"2023-11-28T11:51:17.880383+0000","last_clean":"2023-11-28T11:51:17.880383+0000","last_became_active":"2023-11-28T11:51:16.884176+0000","last_became_peered":"2023-11-28T11:51:16.884176+0000","last_unstale":"2023-11-28T11:51:17.880383+0000","last_undegraded":"2023-11-28T11:51:17.880383+0000","last_fullsized":"2023-11-28T11:51:17.880383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093566+0000","last_change":"2023-11-28T11:51:16.488286+0000","last_active":"2023-11-28T11:51:19.093566+0000","last_peered":"2023-11-28T11:51:19.093566+0000","last_clean":"2023-11-28T11:51:19.093566+0000","last_became_active":"2023-11-28T11:51:16.487987+0000","last_became_peered":"2023-11-28T11:51:16.487987+0000","last_unstale":"2023-11-28T11:51:19.093566+0000","last_undegraded":"2023-11-28T11:51:19.093566+0000","last_fullsized":"2023-11-28T11:51:19.093566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:39:02.250990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880976+0000","last_change":"2023-11-28T11:51:16.888775+0000","last_active":"2023-11-28T11:51:17.880976+0000","last_peered":"2023-11-28T11:51:17.880976+0000","last_clean":"2023-11-28T11:51:17.880976+0000","last_became_active":"2023-11-28T11:51:16.888485+0000","last_became_peered":"2023-11-28T11:51:16.888485+0000","last_unstale":"2023-11-28T11:51:17.880976+0000","last_undegraded":"2023-11-28T11:51:17.880976+0000","last_fullsized":"2023-11-28T11:51:17.880976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:55.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093487+0000","last_change":"2023-11-28T11:51:16.888856+0000","last_active":"2023-11-28T11:51:19.093487+0000","last_peered":"2023-11-28T11:51:19.093487+0000","last_clean":"2023-11-28T11:51:19.093487+0000","last_became_active":"2023-11-28T11:51:16.887628+0000","last_became_peered":"2023-11-28T11:51:16.887628+0000","last_unstale":"2023-11-28T11:51:19.093487+0000","last_undegraded":"2023-11-28T11:51:19.093487+0000","last_fullsized":"2023-11-28T11:51:19.093487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880202+0000","last_change":"2023-11-28T11:51:16.887897+0000","last_active":"2023-11-28T11:51:17.880202+0000","last_peered":"2023-11-28T11:51:17.880202+0000","last_clean":"2023-11-28T11:51:17.880202+0000","last_became_active":"2023-11-28T11:51:16.887663+0000","last_became_peered":"2023-11-28T11:51:16.887663+0000","last_unstale":"2023-11-28T11:51:17.880202+0000","last_undegraded":"2023-11-28T11:51:17.880202+0000","last_fullsized":"2023-11-28T11:51:17.880202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881169+0000","last_change":"2023-11-28T11:51:16.891805+0000","last_active":"2023-11-28T11:51:17.881169+0000","last_peered":"2023-11-28T11:51:17.881169+0000","last_clean":"2023-11-28T11:51:17.881169+0000","last_became_active":"2023-11-28T11:51:16.891091+0000","last_became_peered":"2023-11-28T11:51:16.891091+0000","last_unstale":"2023-11-28T11:51:17.881169+0000","last_undegraded":"2023-11-28T11:51:17.881169+0000","last_fullsized":"2023-11-28T11:51:17.881169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880475+0000","last_change":"2023-11-28T11:51:16.897807+0000","last_active":"2023-11-28T11:51:17.880475+0000","last_peered":"2023-11-28T11:51:17.880475+0000","last_clean":"2023-11-28T11:51:17.880475+0000","last_became_active":"2023-11-28T11:51:16.897481+0000","last_became_peered":"2023-11-28T11:51:16.897481+0000","last_unstale":"2023-11-28T11:51:17.880475+0000","last_undegraded":"2023-11-28T11:51:17.880475+0000","last_fullsized":"2023-11-28T11:51:17.880475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093497+0000","last_change":"2023-11-28T11:51:16.886025+0000","last_active":"2023-11-28T11:51:19.093497+0000","last_peered":"2023-11-28T11:51:19.093497+0000","last_clean":"2023-11-28T11:51:19.093497+0000","last_became_active":"2023-11-28T11:51:16.885791+0000","last_became_peered":"2023-11-28T11:51:16.885791+0000","last_unstale":"2023-11-28T11:51:19.093497+0000","last_undegraded":"2023-11-28T11:51:19.093497+0000","last_fullsized":"2023-11-28T11:51:19.093497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9092,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362748,"statfs":{"total":96636764160,"available":96627453952,"internally_reserved":0,"allocated":847872,"data_stored":674887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9064,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362776,"statfs":{"total":96636764160,"available":96627482624,"internally_reserved":0,"allocated":819200,"data_stored":673123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8036,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363804,"statfs":{"total":96636764160,"available":96628535296,"internally_reserved":0,"allocated":225280,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8040,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363800,"statfs":{"total":96636764160,"available":96628531200,"internally_reserved":0,"allocated":229376,"data_stored":82845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":84359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7976,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363864,"statfs":{"total":96636764160,"available":96628596736,"internally_reserved":0,"allocated":229376,"data_stored":82405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:51:25.546 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:51:25.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 Processor -- start 2023-11-28T11:51:25.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 -- start start 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff504071040 0x7ff504071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff50406dee0 0x7ff50406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff50406cdc0 0x7ff50406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff504071950 con 0x7ff50406cdc0 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50415d060 con 0x7ff50406dee0 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.701+0000 7ff509be5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415d340 con 0x7ff504071040 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff50406cdc0 0x7ff50406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff50406cdc0 0x7ff50406d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34876/0 (socket says 172.21.15.144:34876) 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 -- 172.21.15.144:0/1553895479 learned_addr learned my addr 172.21.15.144:0/1553895479 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff5037fe700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff504071040 0x7ff504071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff502ffd700 1 --2- 172.21.15.144:0/1553895479 >> v2:172.21.15.177:3300/0 conn(0x7ff50406dee0 0x7ff50406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 -- 172.21.15.144:0/1553895479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff504071040 msgr2=0x7ff504071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff504071040 0x7ff504071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 -- 172.21.15.144:0/1553895479 >> v2:172.21.15.177:3300/0 conn(0x7ff50406dee0 msgr2=0x7ff50406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 --2- 172.21.15.144:0/1553895479 >> v2:172.21.15.177:3300/0 conn(0x7ff50406dee0 0x7ff50406e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 -- 172.21.15.144:0/1553895479 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50415d620 con 0x7ff50406cdc0 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff5037fe700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff504071040 0x7ff504071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff503fff700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff50406cdc0 0x7ff50406d240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff50409e140 tx=0x7ff4fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1406d8b14ccf4c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff5027fc700 1 -- 172.21.15.144:0/1553895479 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00e070 con 0x7ff50406cdc0 2023-11-28T11:51:25.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff5027fc700 1 -- 172.21.15.144:0/1553895479 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00b9c0 con 0x7ff50406cdc0 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.702+0000 7ff5027fc700 1 -- 172.21.15.144:0/1553895479 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00bd70 con 0x7ff50406cdc0 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 -- 172.21.15.144:0/1553895479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff50406cdc0 msgr2=0x7ff50406d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff50406cdc0 0x7ff50406d240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff50409e140 tx=0x7ff4fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 -- 172.21.15.144:0/1553895479 shutdown_connections 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff50406cdc0 0x7ff50406d240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 --2- 172.21.15.144:0/1553895479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff504071040 0x7ff504071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 --2- 172.21.15.144:0/1553895479 >> v2:172.21.15.177:3300/0 conn(0x7ff50406dee0 0x7ff50406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 -- 172.21.15.144:0/1553895479 >> 172.21.15.144:0/1553895479 conn(0x7ff5040d0000 msgr2=0x7ff5040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 -- 172.21.15.144:0/1553895479 shutdown_connections 2023-11-28T11:51:25.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.703+0000 7ff509be5700 1 -- 172.21.15.144:0/1553895479 wait complete. 2023-11-28T11:51:25.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.704+0000 7ff509be5700 1 Processor -- start 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.704+0000 7ff509be5700 1 -- start start 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff509be5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff50406cdc0 0x7ff50415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff509be5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff50406dee0 0x7ff5040c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff509be5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 0x7ff504153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff509be5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040c3fe0 con 0x7ff504071040 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff509be5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040c4e00 con 0x7ff50406dee0 2023-11-28T11:51:25.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff509be5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040c41f0 con 0x7ff50406cdc0 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 0x7ff504153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 0x7ff504153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34882/0 (socket says 172.21.15.144:34882) 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 -- 172.21.15.144:0/810644987 learned_addr learned my addr 172.21.15.144:0/810644987 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff5037fe700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff50406cdc0 0x7ff50415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff502ffd700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff50406dee0 0x7ff5040c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 -- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff50406cdc0 msgr2=0x7ff50415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff50406cdc0 0x7ff50415b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 -- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff50406dee0 msgr2=0x7ff5040c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff50406dee0 0x7ff5040c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 -- 172.21.15.144:0/810644987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc009b30 con 0x7ff504071040 2023-11-28T11:51:25.712 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff502ffd700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff50406dee0 0x7ff5040c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff503fff700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 0x7ff504153080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff50406c7b0 tx=0x7ff4fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.705+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc004040 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.706+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0041c0 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.706+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50406e890 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.706+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc004570 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.706+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50415d900 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.706+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50415ccd0 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.707+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff4fc02c070 con 0x7ff504071040 2023-11-28T11:51:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.707+0000 7ff4fbfff700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff4e40682f0 0x7ff4e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:25.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.707+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4fc086eb0 con 0x7ff504071040 2023-11-28T11:51:25.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.707+0000 7ff5037fe700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff4e40682f0 0x7ff4e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:25.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.708+0000 7ff5037fe700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff4e40682f0 0x7ff4e406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c940 tx=0x7ff4ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:25.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.711+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff4fc053710 con 0x7ff504071040 2023-11-28T11:51:25.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.888+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff50409de50 con 0x7ff4e40682f0 2023-11-28T11:51:25.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.896+0000 7ff4fbfff700 1 -- 172.21.15.144:0/810644987 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286772 (secure 0 0 0) 0x7ff50409de50 con 0x7ff4e40682f0 2023-11-28T11:51:25.900 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:25.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff4e40682f0 msgr2=0x7ff4e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff4e40682f0 0x7ff4e406a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c940 tx=0x7ff4ec008040 comp rx=0 tx=0).stop 2023-11-28T11:51:25.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 msgr2=0x7ff504153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:25.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 0x7ff504153080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff50406c7b0 tx=0x7ff4fc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 shutdown_connections 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff504071040 0x7ff504153080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff4e40682f0 0x7ff4e406a7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff50406cdc0 0x7ff50415b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 --2- 172.21.15.144:0/810644987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff50406dee0 0x7ff5040c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 >> 172.21.15.144:0/810644987 conn(0x7ff5040d0000 msgr2=0x7ff50406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 shutdown_connections 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:25.899+0000 7ff509be5700 1 -- 172.21.15.144:0/810644987 wait complete. 2023-11-28T11:51:25.903 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:51:25.913 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T11:51:24.484331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":66080,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":63040,"kb_avail":754908640,"statfs":{"total":773094113280,"available":773026447360,"internally_reserved":0,"allocated":3047424,"data_stored":1845464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":64552960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003067"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092580+0000","last_change":"2023-11-28T11:51:16.886968+0000","last_active":"2023-11-28T11:51:19.092580+0000","last_peered":"2023-11-28T11:51:19.092580+0000","last_clean":"2023-11-28T11:51:19.092580+0000","last_became_active":"2023-11-28T11:51:16.885939+0000","last_became_peered":"2023-11-28T11:51:16.885939+0000","last_unstale":"2023-11-28T11:51:19.092580+0000","last_undegraded":"2023-11-28T11:51:19.092580+0000","last_fullsized":"2023-11-28T11:51:19.092580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:14.596675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879462+0000","last_change":"2023-11-28T11:51:16.887419+0000","last_active":"2023-11-28T11:51:17.879462+0000","last_peered":"2023-11-28T11:51:17.879462+0000","last_clean":"2023-11-28T11:51:17.879462+0000","last_became_active":"2023-11-28T11:51:16.887120+0000","last_became_peered":"2023-11-28T11:51:16.887120+0000","last_unstale":"2023-11-28T11:51:17.879462+0000","last_undegraded":"2023-11-28T11:51:17.879462+0000","last_fullsized":"2023-11-28T11:51:17.879462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881469+0000","last_change":"2023-11-28T11:51:16.899556+0000","last_active":"2023-11-28T11:51:17.881469+0000","last_peered":"2023-11-28T11:51:17.881469+0000","last_clean":"2023-11-28T11:51:17.881469+0000","last_became_active":"2023-11-28T11:51:16.899088+0000","last_became_peered":"2023-11-28T11:51:16.899088+0000","last_unstale":"2023-11-28T11:51:17.881469+0000","last_undegraded":"2023-11-28T11:51:17.881469+0000","last_fullsized":"2023-11-28T11:51:17.881469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:23.190053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093312+0000","last_change":"2023-11-28T11:51:16.898501+0000","last_active":"2023-11-28T11:51:19.093312+0000","last_peered":"2023-11-28T11:51:19.093312+0000","last_clean":"2023-11-28T11:51:19.093312+0000","last_became_active":"2023-11-28T11:51:16.898263+0000","last_became_peered":"2023-11-28T11:51:16.898263+0000","last_unstale":"2023-11-28T11:51:19.093312+0000","last_undegraded":"2023-11-28T11:51:19.093312+0000","last_fullsized":"2023-11-28T11:51:19.093312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093023+0000","last_change":"2023-11-28T11:51:16.885205+0000","last_active":"2023-11-28T11:51:19.093023+0000","last_peered":"2023-11-28T11:51:19.093023+0000","last_clean":"2023-11-28T11:51:19.093023+0000","last_became_active":"2023-11-28T11:51:16.884964+0000","last_became_peered":"2023-11-28T11:51:16.884964+0000","last_unstale":"2023-11-28T11:51:19.093023+0000","last_undegraded":"2023-11-28T11:51:19.093023+0000","last_fullsized":"2023-11-28T11:51:19.093023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:36.307669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881261+0000","last_change":"2023-11-28T11:51:16.899331+0000","last_active":"2023-11-28T11:51:17.881261+0000","last_peered":"2023-11-28T11:51:17.881261+0000","last_clean":"2023-11-28T11:51:17.881261+0000","last_became_active":"2023-11-28T11:51:16.899026+0000","last_became_peered":"2023-11-28T11:51:16.899026+0000","last_unstale":"2023-11-28T11:51:17.881261+0000","last_undegraded":"2023-11-28T11:51:17.881261+0000","last_fullsized":"2023-11-28T11:51:17.881261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:40.625446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093069+0000","last_change":"2023-11-28T11:51:16.895279+0000","last_active":"2023-11-28T11:51:19.093069+0000","last_peered":"2023-11-28T11:51:19.093069+0000","last_clean":"2023-11-28T11:51:19.093069+0000","last_became_active":"2023-11-28T11:51:16.894949+0000","last_became_peered":"2023-11-28T11:51:16.894949+0000","last_unstale":"2023-11-28T11:51:19.093069+0000","last_undegraded":"2023-11-28T11:51:19.093069+0000","last_fullsized":"2023-11-28T11:51:19.093069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.882353+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.882010+0000","last_became_peered":"2023-11-28T11:51:16.882010+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093331+0000","last_change":"2023-11-28T11:51:16.885987+0000","last_active":"2023-11-28T11:51:19.093331+0000","last_peered":"2023-11-28T11:51:19.093331+0000","last_clean":"2023-11-28T11:51:19.093331+0000","last_became_active":"2023-11-28T11:51:16.885744+0000","last_became_peered":"2023-11-28T11:51:16.885744+0000","last_unstale":"2023-11-28T11:51:19.093331+0000","last_undegraded":"2023-11-28T11:51:19.093331+0000","last_fullsized":"2023-11-28T11:51:19.093331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:20:59.112675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879642+0000","last_change":"2023-11-28T11:51:16.898873+0000","last_active":"2023-11-28T11:51:17.879642+0000","last_peered":"2023-11-28T11:51:17.879642+0000","last_clean":"2023-11-28T11:51:17.879642+0000","last_became_active":"2023-11-28T11:51:16.898567+0000","last_became_peered":"2023-11-28T11:51:16.898567+0000","last_unstale":"2023-11-28T11:51:17.879642+0000","last_undegraded":"2023-11-28T11:51:17.879642+0000","last_fullsized":"2023-11-28T11:51:17.879642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881611+0000","last_change":"2023-11-28T11:51:16.892348+0000","last_active":"2023-11-28T11:51:17.881611+0000","last_peered":"2023-11-28T11:51:17.881611+0000","last_clean":"2023-11-28T11:51:17.881611+0000","last_became_active":"2023-11-28T11:51:16.891451+0000","last_became_peered":"2023-11-28T11:51:16.891451+0000","last_unstale":"2023-11-28T11:51:17.881611+0000","last_undegraded":"2023-11-28T11:51:17.881611+0000","last_fullsized":"2023-11-28T11:51:17.881611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:37.035204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092706+0000","last_change":"2023-11-28T11:51:16.893555+0000","last_active":"2023-11-28T11:51:19.092706+0000","last_peered":"2023-11-28T11:51:19.092706+0000","last_clean":"2023-11-28T11:51:19.092706+0000","last_became_active":"2023-11-28T11:51:16.893296+0000","last_became_peered":"2023-11-28T11:51:16.893296+0000","last_unstale":"2023-11-28T11:51:19.092706+0000","last_undegraded":"2023-11-28T11:51:19.092706+0000","last_fullsized":"2023-11-28T11:51:19.092706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:13.298585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092713+0000","last_change":"2023-11-28T11:51:16.885357+0000","last_active":"2023-11-28T11:51:19.092713+0000","last_peered":"2023-11-28T11:51:19.092713+0000","last_clean":"2023-11-28T11:51:19.092713+0000","last_became_active":"2023-11-28T11:51:16.885098+0000","last_became_peered":"2023-11-28T11:51:16.885098+0000","last_unstale":"2023-11-28T11:51:19.092713+0000","last_undegraded":"2023-11-28T11:51:19.092713+0000","last_fullsized":"2023-11-28T11:51:19.092713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:18.850790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880260+0000","last_change":"2023-11-28T11:51:16.882443+0000","last_active":"2023-11-28T11:51:17.880260+0000","last_peered":"2023-11-28T11:51:17.880260+0000","last_clean":"2023-11-28T11:51:17.880260+0000","last_became_active":"2023-11-28T11:51:16.882164+0000","last_became_peered":"2023-11-28T11:51:16.882164+0000","last_unstale":"2023-11-28T11:51:17.880260+0000","last_undegraded":"2023-11-28T11:51:17.880260+0000","last_fullsized":"2023-11-28T11:51:17.880260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093567+0000","last_change":"2023-11-28T11:51:16.895797+0000","last_active":"2023-11-28T11:51:19.093567+0000","last_peered":"2023-11-28T11:51:19.093567+0000","last_clean":"2023-11-28T11:51:19.093567+0000","last_became_active":"2023-11-28T11:51:16.895517+0000","last_became_peered":"2023-11-28T11:51:16.895517+0000","last_unstale":"2023-11-28T11:51:19.093567+0000","last_undegraded":"2023-11-28T11:51:19.093567+0000","last_fullsized":"2023-11-28T11:51:19.093567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881087+0000","last_change":"2023-11-28T11:51:16.899351+0000","last_active":"2023-11-28T11:51:17.881087+0000","last_peered":"2023-11-28T11:51:17.881087+0000","last_clean":"2023-11-28T11:51:17.881087+0000","last_became_active":"2023-11-28T11:51:16.899044+0000","last_became_peered":"2023-11-28T11:51:16.899044+0000","last_unstale":"2023-11-28T11:51:17.881087+0000","last_undegraded":"2023-11-28T11:51:17.881087+0000","last_fullsized":"2023-11-28T11:51:17.881087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:09.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093414+0000","last_change":"2023-11-28T11:51:16.885961+0000","last_active":"2023-11-28T11:51:19.093414+0000","last_peered":"2023-11-28T11:51:19.093414+0000","last_clean":"2023-11-28T11:51:19.093414+0000","last_became_active":"2023-11-28T11:51:16.885709+0000","last_became_peered":"2023-11-28T11:51:16.885709+0000","last_unstale":"2023-11-28T11:51:19.093414+0000","last_undegraded":"2023-11-28T11:51:19.093414+0000","last_fullsized":"2023-11-28T11:51:19.093414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:13.291091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881602+0000","last_change":"2023-11-28T11:51:16.893897+0000","last_active":"2023-11-28T11:51:17.881602+0000","last_peered":"2023-11-28T11:51:17.881602+0000","last_clean":"2023-11-28T11:51:17.881602+0000","last_became_active":"2023-11-28T11:51:16.893541+0000","last_became_peered":"2023-11-28T11:51:16.893541+0000","last_unstale":"2023-11-28T11:51:17.881602+0000","last_undegraded":"2023-11-28T11:51:17.881602+0000","last_fullsized":"2023-11-28T11:51:17.881602+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092888+0000","last_change":"2023-11-28T11:51:16.887327+0000","last_active":"2023-11-28T11:51:19.092888+0000","last_peered":"2023-11-28T11:51:19.092888+0000","last_clean":"2023-11-28T11:51:19.092888+0000","last_became_active":"2023-11-28T11:51:16.887062+0000","last_became_peered":"2023-11-28T11:51:16.887062+0000","last_unstale":"2023-11-28T11:51:19.092888+0000","last_undegraded":"2023-11-28T11:51:19.092888+0000","last_fullsized":"2023-11-28T11:51:19.092888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:49.047566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881480+0000","last_change":"2023-11-28T11:51:16.889122+0000","last_active":"2023-11-28T11:51:17.881480+0000","last_peered":"2023-11-28T11:51:17.881480+0000","last_clean":"2023-11-28T11:51:17.881480+0000","last_became_active":"2023-11-28T11:51:16.888849+0000","last_became_peered":"2023-11-28T11:51:16.888849+0000","last_unstale":"2023-11-28T11:51:17.881480+0000","last_undegraded":"2023-11-28T11:51:17.881480+0000","last_fullsized":"2023-11-28T11:51:17.881480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093486+0000","last_change":"2023-11-28T11:51:16.895113+0000","last_active":"2023-11-28T11:51:19.093486+0000","last_peered":"2023-11-28T11:51:19.093486+0000","last_clean":"2023-11-28T11:51:19.093486+0000","last_became_active":"2023-11-28T11:51:16.894793+0000","last_became_peered":"2023-11-28T11:51:16.894793+0000","last_unstale":"2023-11-28T11:51:19.093486+0000","last_undegraded":"2023-11-28T11:51:19.093486+0000","last_fullsized":"2023-11-28T11:51:19.093486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898452+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897946+0000","last_became_peered":"2023-11-28T11:51:16.897946+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:20.222278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880183+0000","last_change":"2023-11-28T11:51:16.898903+0000","last_active":"2023-11-28T11:51:17.880183+0000","last_peered":"2023-11-28T11:51:17.880183+0000","last_clean":"2023-11-28T11:51:17.880183+0000","last_became_active":"2023-11-28T11:51:16.898719+0000","last_became_peered":"2023-11-28T11:51:16.898719+0000","last_unstale":"2023-11-28T11:51:17.880183+0000","last_undegraded":"2023-11-28T11:51:17.880183+0000","last_fullsized":"2023-11-28T11:51:17.880183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093156+0000","last_change":"2023-11-28T11:51:16.895329+0000","last_active":"2023-11-28T11:51:19.093156+0000","last_peered":"2023-11-28T11:51:19.093156+0000","last_clean":"2023-11-28T11:51:19.093156+0000","last_became_active":"2023-11-28T11:51:16.894998+0000","last_became_peered":"2023-11-28T11:51:16.894998+0000","last_unstale":"2023-11-28T11:51:19.093156+0000","last_undegraded":"2023-11-28T11:51:19.093156+0000","last_fullsized":"2023-11-28T11:51:19.093156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093690+0000","last_change":"2023-11-28T11:51:16.889063+0000","last_active":"2023-11-28T11:51:19.093690+0000","last_peered":"2023-11-28T11:51:19.093690+0000","last_clean":"2023-11-28T11:51:19.093690+0000","last_became_active":"2023-11-28T11:51:16.888711+0000","last_became_peered":"2023-11-28T11:51:16.888711+0000","last_unstale":"2023-11-28T11:51:19.093690+0000","last_undegraded":"2023-11-28T11:51:19.093690+0000","last_fullsized":"2023-11-28T11:51:19.093690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:11.196742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093545+0000","last_change":"2023-11-28T11:51:16.882290+0000","last_active":"2023-11-28T11:51:19.093545+0000","last_peered":"2023-11-28T11:51:19.093545+0000","last_clean":"2023-11-28T11:51:19.093545+0000","last_became_active":"2023-11-28T11:51:16.882049+0000","last_became_peered":"2023-11-28T11:51:16.882049+0000","last_unstale":"2023-11-28T11:51:19.093545+0000","last_undegraded":"2023-11-28T11:51:19.093545+0000","last_fullsized":"2023-11-28T11:51:19.093545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:41.991365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879435+0000","last_change":"2023-11-28T11:51:16.896720+0000","last_active":"2023-11-28T11:51:17.879435+0000","last_peered":"2023-11-28T11:51:17.879435+0000","last_clean":"2023-11-28T11:51:17.879435+0000","last_became_active":"2023-11-28T11:51:16.896429+0000","last_became_peered":"2023-11-28T11:51:16.896429+0000","last_unstale":"2023-11-28T11:51:17.879435+0000","last_undegraded":"2023-11-28T11:51:17.879435+0000","last_fullsized":"2023-11-28T11:51:17.879435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881315+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:17.881315+0000","last_peered":"2023-11-28T11:51:17.881315+0000","last_clean":"2023-11-28T11:51:17.881315+0000","last_became_active":"2023-11-28T11:51:16.482973+0000","last_became_peered":"2023-11-28T11:51:16.482973+0000","last_unstale":"2023-11-28T11:51:17.881315+0000","last_undegraded":"2023-11-28T11:51:17.881315+0000","last_fullsized":"2023-11-28T11:51:17.881315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880643+0000","last_change":"2023-11-28T11:51:16.887525+0000","last_active":"2023-11-28T11:51:17.880643+0000","last_peered":"2023-11-28T11:51:17.880643+0000","last_clean":"2023-11-28T11:51:17.880643+0000","last_became_active":"2023-11-28T11:51:16.887247+0000","last_became_peered":"2023-11-28T11:51:16.887247+0000","last_unstale":"2023-11-28T11:51:17.880643+0000","last_undegraded":"2023-11-28T11:51:17.880643+0000","last_fullsized":"2023-11-28T11:51:17.880643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093032+0000","last_change":"2023-11-28T11:51:16.481846+0000","last_active":"2023-11-28T11:51:19.093032+0000","last_peered":"2023-11-28T11:51:19.093032+0000","last_clean":"2023-11-28T11:51:19.093032+0000","last_became_active":"2023-11-28T11:51:16.481653+0000","last_became_peered":"2023-11-28T11:51:16.481653+0000","last_unstale":"2023-11-28T11:51:19.093032+0000","last_undegraded":"2023-11-28T11:51:19.093032+0000","last_fullsized":"2023-11-28T11:51:19.093032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:21.555920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093520+0000","last_change":"2023-11-28T11:51:16.895323+0000","last_active":"2023-11-28T11:51:19.093520+0000","last_peered":"2023-11-28T11:51:19.093520+0000","last_clean":"2023-11-28T11:51:19.093520+0000","last_became_active":"2023-11-28T11:51:16.895030+0000","last_became_peered":"2023-11-28T11:51:16.895030+0000","last_unstale":"2023-11-28T11:51:19.093520+0000","last_undegraded":"2023-11-28T11:51:19.093520+0000","last_fullsized":"2023-11-28T11:51:19.093520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:59.340661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880302+0000","last_change":"2023-11-28T11:51:16.488974+0000","last_active":"2023-11-28T11:51:17.880302+0000","last_peered":"2023-11-28T11:51:17.880302+0000","last_clean":"2023-11-28T11:51:17.880302+0000","last_became_active":"2023-11-28T11:51:16.488697+0000","last_became_peered":"2023-11-28T11:51:16.488697+0000","last_unstale":"2023-11-28T11:51:17.880302+0000","last_undegraded":"2023-11-28T11:51:17.880302+0000","last_fullsized":"2023-11-28T11:51:17.880302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093841+0000","last_change":"2023-11-28T11:51:16.888129+0000","last_active":"2023-11-28T11:51:19.093841+0000","last_peered":"2023-11-28T11:51:19.093841+0000","last_clean":"2023-11-28T11:51:19.093841+0000","last_became_active":"2023-11-28T11:51:16.887848+0000","last_became_peered":"2023-11-28T11:51:16.887848+0000","last_unstale":"2023-11-28T11:51:19.093841+0000","last_undegraded":"2023-11-28T11:51:19.093841+0000","last_fullsized":"2023-11-28T11:51:19.093841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:42.239688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093488+0000","last_change":"2023-11-28T11:51:16.488283+0000","last_active":"2023-11-28T11:51:19.093488+0000","last_peered":"2023-11-28T11:51:19.093488+0000","last_clean":"2023-11-28T11:51:19.093488+0000","last_became_active":"2023-11-28T11:51:16.487994+0000","last_became_peered":"2023-11-28T11:51:16.487994+0000","last_unstale":"2023-11-28T11:51:19.093488+0000","last_undegraded":"2023-11-28T11:51:19.093488+0000","last_fullsized":"2023-11-28T11:51:19.093488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:04.545703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880313+0000","last_change":"2023-11-28T11:51:16.891442+0000","last_active":"2023-11-28T11:51:17.880313+0000","last_peered":"2023-11-28T11:51:17.880313+0000","last_clean":"2023-11-28T11:51:17.880313+0000","last_became_active":"2023-11-28T11:51:16.891180+0000","last_became_peered":"2023-11-28T11:51:16.891180+0000","last_unstale":"2023-11-28T11:51:17.880313+0000","last_undegraded":"2023-11-28T11:51:17.880313+0000","last_fullsized":"2023-11-28T11:51:17.880313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:08.023450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879563+0000","last_change":"2023-11-28T11:51:16.484401+0000","last_active":"2023-11-28T11:51:17.879563+0000","last_peered":"2023-11-28T11:51:17.879563+0000","last_clean":"2023-11-28T11:51:17.879563+0000","last_became_active":"2023-11-28T11:51:16.484103+0000","last_became_peered":"2023-11-28T11:51:16.484103+0000","last_unstale":"2023-11-28T11:51:17.879563+0000","last_undegraded":"2023-11-28T11:51:17.879563+0000","last_fullsized":"2023-11-28T11:51:17.879563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880312+0000","last_change":"2023-11-28T11:51:16.898526+0000","last_active":"2023-11-28T11:51:17.880312+0000","last_peered":"2023-11-28T11:51:17.880312+0000","last_clean":"2023-11-28T11:51:17.880312+0000","last_became_active":"2023-11-28T11:51:16.897658+0000","last_became_peered":"2023-11-28T11:51:16.897658+0000","last_unstale":"2023-11-28T11:51:17.880312+0000","last_undegraded":"2023-11-28T11:51:17.880312+0000","last_fullsized":"2023-11-28T11:51:17.880312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093666+0000","last_change":"2023-11-28T11:51:16.484674+0000","last_active":"2023-11-28T11:51:19.093666+0000","last_peered":"2023-11-28T11:51:19.093666+0000","last_clean":"2023-11-28T11:51:19.093666+0000","last_became_active":"2023-11-28T11:51:16.484350+0000","last_became_peered":"2023-11-28T11:51:16.484350+0000","last_unstale":"2023-11-28T11:51:19.093666+0000","last_undegraded":"2023-11-28T11:51:19.093666+0000","last_fullsized":"2023-11-28T11:51:19.093666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:13.351037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093718+0000","last_change":"2023-11-28T11:51:16.889523+0000","last_active":"2023-11-28T11:51:19.093718+0000","last_peered":"2023-11-28T11:51:19.093718+0000","last_clean":"2023-11-28T11:51:19.093718+0000","last_became_active":"2023-11-28T11:51:16.889323+0000","last_became_peered":"2023-11-28T11:51:16.889323+0000","last_unstale":"2023-11-28T11:51:19.093718+0000","last_undegraded":"2023-11-28T11:51:19.093718+0000","last_fullsized":"2023-11-28T11:51:19.093718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881658+0000","last_change":"2023-11-28T11:51:16.482345+0000","last_active":"2023-11-28T11:51:17.881658+0000","last_peered":"2023-11-28T11:51:17.881658+0000","last_clean":"2023-11-28T11:51:17.881658+0000","last_became_active":"2023-11-28T11:51:16.482106+0000","last_became_peered":"2023-11-28T11:51:16.482106+0000","last_unstale":"2023-11-28T11:51:17.881658+0000","last_undegraded":"2023-11-28T11:51:17.881658+0000","last_fullsized":"2023-11-28T11:51:17.881658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881274+0000","last_change":"2023-11-28T11:51:16.890129+0000","last_active":"2023-11-28T11:51:17.881274+0000","last_peered":"2023-11-28T11:51:17.881274+0000","last_clean":"2023-11-28T11:51:17.881274+0000","last_became_active":"2023-11-28T11:51:16.889849+0000","last_became_peered":"2023-11-28T11:51:16.889849+0000","last_unstale":"2023-11-28T11:51:17.881274+0000","last_undegraded":"2023-11-28T11:51:17.881274+0000","last_fullsized":"2023-11-28T11:51:17.881274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093375+0000","last_change":"2023-11-28T11:51:16.485194+0000","last_active":"2023-11-28T11:51:19.093375+0000","last_peered":"2023-11-28T11:51:19.093375+0000","last_clean":"2023-11-28T11:51:19.093375+0000","last_became_active":"2023-11-28T11:51:16.484873+0000","last_became_peered":"2023-11-28T11:51:16.484873+0000","last_unstale":"2023-11-28T11:51:19.093375+0000","last_undegraded":"2023-11-28T11:51:19.093375+0000","last_fullsized":"2023-11-28T11:51:19.093375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880521+0000","last_change":"2023-11-28T11:51:16.887321+0000","last_active":"2023-11-28T11:51:17.880521+0000","last_peered":"2023-11-28T11:51:17.880521+0000","last_clean":"2023-11-28T11:51:17.880521+0000","last_became_active":"2023-11-28T11:51:16.886989+0000","last_became_peered":"2023-11-28T11:51:16.886989+0000","last_unstale":"2023-11-28T11:51:17.880521+0000","last_undegraded":"2023-11-28T11:51:17.880521+0000","last_fullsized":"2023-11-28T11:51:17.880521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:17.842274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093431+0000","last_change":"2023-11-28T11:51:16.486311+0000","last_active":"2023-11-28T11:51:19.093431+0000","last_peered":"2023-11-28T11:51:19.093431+0000","last_clean":"2023-11-28T11:51:19.093431+0000","last_became_active":"2023-11-28T11:51:16.485952+0000","last_became_peered":"2023-11-28T11:51:16.485952+0000","last_unstale":"2023-11-28T11:51:19.093431+0000","last_undegraded":"2023-11-28T11:51:19.093431+0000","last_fullsized":"2023-11-28T11:51:19.093431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:06:13.615057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879568+0000","last_change":"2023-11-28T11:51:16.896713+0000","last_active":"2023-11-28T11:51:17.879568+0000","last_peered":"2023-11-28T11:51:17.879568+0000","last_clean":"2023-11-28T11:51:17.879568+0000","last_became_active":"2023-11-28T11:51:16.896490+0000","last_became_peered":"2023-11-28T11:51:16.896490+0000","last_unstale":"2023-11-28T11:51:17.879568+0000","last_undegraded":"2023-11-28T11:51:17.879568+0000","last_fullsized":"2023-11-28T11:51:17.879568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:31.779508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093447+0000","last_change":"2023-11-28T11:51:16.485718+0000","last_active":"2023-11-28T11:51:19.093447+0000","last_peered":"2023-11-28T11:51:19.093447+0000","last_clean":"2023-11-28T11:51:19.093447+0000","last_became_active":"2023-11-28T11:51:16.485279+0000","last_became_peered":"2023-11-28T11:51:16.485279+0000","last_unstale":"2023-11-28T11:51:19.093447+0000","last_undegraded":"2023-11-28T11:51:19.093447+0000","last_fullsized":"2023-11-28T11:51:19.093447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093289+0000","last_change":"2023-11-28T11:51:16.898233+0000","last_active":"2023-11-28T11:51:19.093289+0000","last_peered":"2023-11-28T11:51:19.093289+0000","last_clean":"2023-11-28T11:51:19.093289+0000","last_became_active":"2023-11-28T11:51:16.897971+0000","last_became_peered":"2023-11-28T11:51:16.897971+0000","last_unstale":"2023-11-28T11:51:19.093289+0000","last_undegraded":"2023-11-28T11:51:19.093289+0000","last_fullsized":"2023-11-28T11:51:19.093289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:59.713494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092990+0000","last_change":"2023-11-28T11:51:16.489032+0000","last_active":"2023-11-28T11:51:19.092990+0000","last_peered":"2023-11-28T11:51:19.092990+0000","last_clean":"2023-11-28T11:51:19.092990+0000","last_became_active":"2023-11-28T11:51:16.488752+0000","last_became_peered":"2023-11-28T11:51:16.488752+0000","last_unstale":"2023-11-28T11:51:19.092990+0000","last_undegraded":"2023-11-28T11:51:19.092990+0000","last_fullsized":"2023-11-28T11:51:19.092990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880771+0000","last_change":"2023-11-28T11:51:16.897679+0000","last_active":"2023-11-28T11:51:17.880771+0000","last_peered":"2023-11-28T11:51:17.880771+0000","last_clean":"2023-11-28T11:51:17.880771+0000","last_became_active":"2023-11-28T11:51:16.897336+0000","last_became_peered":"2023-11-28T11:51:16.897336+0000","last_unstale":"2023-11-28T11:51:17.880771+0000","last_undegraded":"2023-11-28T11:51:17.880771+0000","last_fullsized":"2023-11-28T11:51:17.880771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:38.898776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093751+0000","last_change":"2023-11-28T11:51:16.485545+0000","last_active":"2023-11-28T11:51:19.093751+0000","last_peered":"2023-11-28T11:51:19.093751+0000","last_clean":"2023-11-28T11:51:19.093751+0000","last_became_active":"2023-11-28T11:51:16.485097+0000","last_became_peered":"2023-11-28T11:51:16.485097+0000","last_unstale":"2023-11-28T11:51:19.093751+0000","last_undegraded":"2023-11-28T11:51:19.093751+0000","last_fullsized":"2023-11-28T11:51:19.093751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:54:41.232205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879198+0000","last_change":"2023-11-28T11:51:16.896139+0000","last_active":"2023-11-28T11:51:17.879198+0000","last_peered":"2023-11-28T11:51:17.879198+0000","last_clean":"2023-11-28T11:51:17.879198+0000","last_became_active":"2023-11-28T11:51:16.895569+0000","last_became_peered":"2023-11-28T11:51:16.895569+0000","last_unstale":"2023-11-28T11:51:17.879198+0000","last_undegraded":"2023-11-28T11:51:17.879198+0000","last_fullsized":"2023-11-28T11:51:17.879198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093685+0000","last_change":"2023-11-28T11:51:16.485744+0000","last_active":"2023-11-28T11:51:19.093685+0000","last_peered":"2023-11-28T11:51:19.093685+0000","last_clean":"2023-11-28T11:51:19.093685+0000","last_became_active":"2023-11-28T11:51:16.485468+0000","last_became_peered":"2023-11-28T11:51:16.485468+0000","last_unstale":"2023-11-28T11:51:19.093685+0000","last_undegraded":"2023-11-28T11:51:19.093685+0000","last_fullsized":"2023-11-28T11:51:19.093685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:08:55.374012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881144+0000","last_change":"2023-11-28T11:51:16.887450+0000","last_active":"2023-11-28T11:51:17.881144+0000","last_peered":"2023-11-28T11:51:17.881144+0000","last_clean":"2023-11-28T11:51:17.881144+0000","last_became_active":"2023-11-28T11:51:16.887190+0000","last_became_peered":"2023-11-28T11:51:16.887190+0000","last_unstale":"2023-11-28T11:51:17.881144+0000","last_undegraded":"2023-11-28T11:51:17.881144+0000","last_fullsized":"2023-11-28T11:51:17.881144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:00.113811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093607+0000","last_change":"2023-11-28T11:51:16.485512+0000","last_active":"2023-11-28T11:51:19.093607+0000","last_peered":"2023-11-28T11:51:19.093607+0000","last_clean":"2023-11-28T11:51:19.093607+0000","last_became_active":"2023-11-28T11:51:16.485186+0000","last_became_peered":"2023-11-28T11:51:16.485186+0000","last_unstale":"2023-11-28T11:51:19.093607+0000","last_undegraded":"2023-11-28T11:51:19.093607+0000","last_fullsized":"2023-11-28T11:51:19.093607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:41.955746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093236+0000","last_change":"2023-11-28T11:51:16.888771+0000","last_active":"2023-11-28T11:51:19.093236+0000","last_peered":"2023-11-28T11:51:19.093236+0000","last_clean":"2023-11-28T11:51:19.093236+0000","last_became_active":"2023-11-28T11:51:16.888179+0000","last_became_peered":"2023-11-28T11:51:16.888179+0000","last_unstale":"2023-11-28T11:51:19.093236+0000","last_undegraded":"2023-11-28T11:51:19.093236+0000","last_fullsized":"2023-11-28T11:51:19.093236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:16.915009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879495+0000","last_change":"2023-11-28T11:51:16.475494+0000","last_active":"2023-11-28T11:51:17.879495+0000","last_peered":"2023-11-28T11:51:17.879495+0000","last_clean":"2023-11-28T11:51:17.879495+0000","last_became_active":"2023-11-28T11:51:16.475189+0000","last_became_peered":"2023-11-28T11:51:16.475189+0000","last_unstale":"2023-11-28T11:51:17.879495+0000","last_undegraded":"2023-11-28T11:51:17.879495+0000","last_fullsized":"2023-11-28T11:51:17.879495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092876+0000","last_change":"2023-11-28T11:51:16.889569+0000","last_active":"2023-11-28T11:51:19.092876+0000","last_peered":"2023-11-28T11:51:19.092876+0000","last_clean":"2023-11-28T11:51:19.092876+0000","last_became_active":"2023-11-28T11:51:16.888349+0000","last_became_peered":"2023-11-28T11:51:16.888349+0000","last_unstale":"2023-11-28T11:51:19.092876+0000","last_undegraded":"2023-11-28T11:51:19.092876+0000","last_fullsized":"2023-11-28T11:51:19.092876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:37.975550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880421+0000","last_change":"2023-11-28T11:51:16.491156+0000","last_active":"2023-11-28T11:51:17.880421+0000","last_peered":"2023-11-28T11:51:17.880421+0000","last_clean":"2023-11-28T11:51:17.880421+0000","last_became_active":"2023-11-28T11:51:16.490844+0000","last_became_peered":"2023-11-28T11:51:16.490844+0000","last_unstale":"2023-11-28T11:51:17.880421+0000","last_undegraded":"2023-11-28T11:51:17.880421+0000","last_fullsized":"2023-11-28T11:51:17.880421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881598+0000","last_change":"2023-11-28T11:51:16.890271+0000","last_active":"2023-11-28T11:51:17.881598+0000","last_peered":"2023-11-28T11:51:17.881598+0000","last_clean":"2023-11-28T11:51:17.881598+0000","last_became_active":"2023-11-28T11:51:16.889985+0000","last_became_peered":"2023-11-28T11:51:16.889985+0000","last_unstale":"2023-11-28T11:51:17.881598+0000","last_undegraded":"2023-11-28T11:51:17.881598+0000","last_fullsized":"2023-11-28T11:51:17.881598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:52:03.275473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093142+0000","last_change":"2023-11-28T11:51:16.482742+0000","last_active":"2023-11-28T11:51:19.093142+0000","last_peered":"2023-11-28T11:51:19.093142+0000","last_clean":"2023-11-28T11:51:19.093142+0000","last_became_active":"2023-11-28T11:51:16.482411+0000","last_became_peered":"2023-11-28T11:51:16.482411+0000","last_unstale":"2023-11-28T11:51:19.093142+0000","last_undegraded":"2023-11-28T11:51:19.093142+0000","last_fullsized":"2023-11-28T11:51:19.093142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881716+0000","last_change":"2023-11-28T11:51:16.893619+0000","last_active":"2023-11-28T11:51:17.881716+0000","last_peered":"2023-11-28T11:51:17.881716+0000","last_clean":"2023-11-28T11:51:17.881716+0000","last_became_active":"2023-11-28T11:51:16.893353+0000","last_became_peered":"2023-11-28T11:51:16.893353+0000","last_unstale":"2023-11-28T11:51:17.881716+0000","last_undegraded":"2023-11-28T11:51:17.881716+0000","last_fullsized":"2023-11-28T11:51:17.881716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:28.799168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881245+0000","last_change":"2023-11-28T11:51:16.480268+0000","last_active":"2023-11-28T11:51:17.881245+0000","last_peered":"2023-11-28T11:51:17.881245+0000","last_clean":"2023-11-28T11:51:17.881245+0000","last_became_active":"2023-11-28T11:51:16.479377+0000","last_became_peered":"2023-11-28T11:51:16.479377+0000","last_unstale":"2023-11-28T11:51:17.881245+0000","last_undegraded":"2023-11-28T11:51:17.881245+0000","last_fullsized":"2023-11-28T11:51:17.881245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881631+0000","last_change":"2023-11-28T11:51:16.890147+0000","last_active":"2023-11-28T11:51:17.881631+0000","last_peered":"2023-11-28T11:51:17.881631+0000","last_clean":"2023-11-28T11:51:17.881631+0000","last_became_active":"2023-11-28T11:51:16.889833+0000","last_became_peered":"2023-11-28T11:51:16.889833+0000","last_unstale":"2023-11-28T11:51:17.881631+0000","last_undegraded":"2023-11-28T11:51:17.881631+0000","last_fullsized":"2023-11-28T11:51:17.881631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880209+0000","last_change":"2023-11-28T11:51:16.478507+0000","last_active":"2023-11-28T11:51:17.880209+0000","last_peered":"2023-11-28T11:51:17.880209+0000","last_clean":"2023-11-28T11:51:17.880209+0000","last_became_active":"2023-11-28T11:51:16.478034+0000","last_became_peered":"2023-11-28T11:51:16.478034+0000","last_unstale":"2023-11-28T11:51:17.880209+0000","last_undegraded":"2023-11-28T11:51:17.880209+0000","last_fullsized":"2023-11-28T11:51:17.880209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:12.291101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093877+0000","last_change":"2023-11-28T11:51:16.885456+0000","last_active":"2023-11-28T11:51:19.093877+0000","last_peered":"2023-11-28T11:51:19.093877+0000","last_clean":"2023-11-28T11:51:19.093877+0000","last_became_active":"2023-11-28T11:51:16.885208+0000","last_became_peered":"2023-11-28T11:51:16.885208+0000","last_unstale":"2023-11-28T11:51:19.093877+0000","last_undegraded":"2023-11-28T11:51:19.093877+0000","last_fullsized":"2023-11-28T11:51:19.093877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:24.014435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880712+0000","last_change":"2023-11-28T11:51:16.483690+0000","last_active":"2023-11-28T11:51:17.880712+0000","last_peered":"2023-11-28T11:51:17.880712+0000","last_clean":"2023-11-28T11:51:17.880712+0000","last_became_active":"2023-11-28T11:51:16.483351+0000","last_became_peered":"2023-11-28T11:51:16.483351+0000","last_unstale":"2023-11-28T11:51:17.880712+0000","last_undegraded":"2023-11-28T11:51:17.880712+0000","last_fullsized":"2023-11-28T11:51:17.880712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877898+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:51:16.877898+0000","last_peered":"2023-11-28T11:51:16.877898+0000","last_clean":"2023-11-28T11:51:16.877898+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:51:16.877898+0000","last_undegraded":"2023-11-28T11:51:16.877898+0000","last_fullsized":"2023-11-28T11:51:16.877898+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880121+0000","last_change":"2023-11-28T11:51:16.897901+0000","last_active":"2023-11-28T11:51:17.880121+0000","last_peered":"2023-11-28T11:51:17.880121+0000","last_clean":"2023-11-28T11:51:17.880121+0000","last_became_active":"2023-11-28T11:51:16.897621+0000","last_became_peered":"2023-11-28T11:51:16.897621+0000","last_unstale":"2023-11-28T11:51:17.880121+0000","last_undegraded":"2023-11-28T11:51:17.880121+0000","last_fullsized":"2023-11-28T11:51:17.880121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:53.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876559+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:51:16.876559+0000","last_peered":"2023-11-28T11:51:16.876559+0000","last_clean":"2023-11-28T11:51:16.876559+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:51:16.876559+0000","last_undegraded":"2023-11-28T11:51:16.876559+0000","last_fullsized":"2023-11-28T11:51:16.876559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:20.007269+0000","last_change":"2023-11-28T11:51:16.486571+0000","last_active":"2023-11-28T11:51:20.007269+0000","last_peered":"2023-11-28T11:51:20.007269+0000","last_clean":"2023-11-28T11:51:20.007269+0000","last_became_active":"2023-11-28T11:51:16.486338+0000","last_became_peered":"2023-11-28T11:51:16.486338+0000","last_unstale":"2023-11-28T11:51:20.007269+0000","last_undegraded":"2023-11-28T11:51:20.007269+0000","last_fullsized":"2023-11-28T11:51:20.007269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:54.333661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881136+0000","last_change":"2023-11-28T11:51:16.889991+0000","last_active":"2023-11-28T11:51:17.881136+0000","last_peered":"2023-11-28T11:51:17.881136+0000","last_clean":"2023-11-28T11:51:17.881136+0000","last_became_active":"2023-11-28T11:51:16.889694+0000","last_became_peered":"2023-11-28T11:51:16.889694+0000","last_unstale":"2023-11-28T11:51:17.881136+0000","last_undegraded":"2023-11-28T11:51:17.881136+0000","last_fullsized":"2023-11-28T11:51:17.881136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.870699+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:51:16.870699+0000","last_peered":"2023-11-28T11:51:16.870699+0000","last_clean":"2023-11-28T11:51:16.870699+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:51:16.870699+0000","last_undegraded":"2023-11-28T11:51:16.870699+0000","last_fullsized":"2023-11-28T11:51:16.870699+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.877332+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:51:16.877332+0000","last_peered":"2023-11-28T11:51:16.877332+0000","last_clean":"2023-11-28T11:51:16.877332+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:51:16.877332+0000","last_undegraded":"2023-11-28T11:51:16.877332+0000","last_fullsized":"2023-11-28T11:51:16.877332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093112+0000","last_change":"2023-11-28T11:51:16.486764+0000","last_active":"2023-11-28T11:51:19.093112+0000","last_peered":"2023-11-28T11:51:19.093112+0000","last_clean":"2023-11-28T11:51:19.093112+0000","last_became_active":"2023-11-28T11:51:16.486448+0000","last_became_peered":"2023-11-28T11:51:16.486448+0000","last_unstale":"2023-11-28T11:51:19.093112+0000","last_undegraded":"2023-11-28T11:51:19.093112+0000","last_fullsized":"2023-11-28T11:51:19.093112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:32:15.029206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093385+0000","last_change":"2023-11-28T11:51:16.888008+0000","last_active":"2023-11-28T11:51:19.093385+0000","last_peered":"2023-11-28T11:51:19.093385+0000","last_clean":"2023-11-28T11:51:19.093385+0000","last_became_active":"2023-11-28T11:51:16.887745+0000","last_became_peered":"2023-11-28T11:51:16.887745+0000","last_unstale":"2023-11-28T11:51:19.093385+0000","last_undegraded":"2023-11-28T11:51:19.093385+0000","last_fullsized":"2023-11-28T11:51:19.093385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:39.903538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.876792+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:51:16.876792+0000","last_peered":"2023-11-28T11:51:16.876792+0000","last_clean":"2023-11-28T11:51:16.876792+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:51:16.876792+0000","last_undegraded":"2023-11-28T11:51:16.876792+0000","last_fullsized":"2023-11-28T11:51:16.876792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092461+0000","last_change":"2023-11-28T11:51:16.484243+0000","last_active":"2023-11-28T11:51:19.092461+0000","last_peered":"2023-11-28T11:51:19.092461+0000","last_clean":"2023-11-28T11:51:19.092461+0000","last_became_active":"2023-11-28T11:51:16.482924+0000","last_became_peered":"2023-11-28T11:51:16.482924+0000","last_unstale":"2023-11-28T11:51:19.092461+0000","last_undegraded":"2023-11-28T11:51:19.092461+0000","last_fullsized":"2023-11-28T11:51:19.092461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:13.589192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093787+0000","last_change":"2023-11-28T11:51:16.878987+0000","last_active":"2023-11-28T11:51:19.093787+0000","last_peered":"2023-11-28T11:51:19.093787+0000","last_clean":"2023-11-28T11:51:19.093787+0000","last_became_active":"2023-11-28T11:51:16.878798+0000","last_became_peered":"2023-11-28T11:51:16.878798+0000","last_unstale":"2023-11-28T11:51:19.093787+0000","last_undegraded":"2023-11-28T11:51:19.093787+0000","last_fullsized":"2023-11-28T11:51:19.093787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:17.727700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873448+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:51:16.873448+0000","last_peered":"2023-11-28T11:51:16.873448+0000","last_clean":"2023-11-28T11:51:16.873448+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:51:16.873448+0000","last_undegraded":"2023-11-28T11:51:16.873448+0000","last_fullsized":"2023-11-28T11:51:16.873448+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880298+0000","last_change":"2023-11-28T11:51:16.476786+0000","last_active":"2023-11-28T11:51:17.880298+0000","last_peered":"2023-11-28T11:51:17.880298+0000","last_clean":"2023-11-28T11:51:17.880298+0000","last_became_active":"2023-11-28T11:51:16.476460+0000","last_became_peered":"2023-11-28T11:51:16.476460+0000","last_unstale":"2023-11-28T11:51:17.880298+0000","last_undegraded":"2023-11-28T11:51:17.880298+0000","last_fullsized":"2023-11-28T11:51:17.880298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:00:53.901576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.094045+0000","last_change":"2023-11-28T11:51:16.890298+0000","last_active":"2023-11-28T11:51:19.094045+0000","last_peered":"2023-11-28T11:51:19.094045+0000","last_clean":"2023-11-28T11:51:19.094045+0000","last_became_active":"2023-11-28T11:51:16.890029+0000","last_became_peered":"2023-11-28T11:51:16.890029+0000","last_unstale":"2023-11-28T11:51:19.094045+0000","last_undegraded":"2023-11-28T11:51:19.094045+0000","last_fullsized":"2023-11-28T11:51:19.094045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:35:42.963230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.873404+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:51:16.873404+0000","last_peered":"2023-11-28T11:51:16.873404+0000","last_clean":"2023-11-28T11:51:16.873404+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:51:16.873404+0000","last_undegraded":"2023-11-28T11:51:16.873404+0000","last_fullsized":"2023-11-28T11:51:16.873404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881318+0000","last_change":"2023-11-28T11:51:16.488913+0000","last_active":"2023-11-28T11:51:17.881318+0000","last_peered":"2023-11-28T11:51:17.881318+0000","last_clean":"2023-11-28T11:51:17.881318+0000","last_became_active":"2023-11-28T11:51:16.488559+0000","last_became_peered":"2023-11-28T11:51:16.488559+0000","last_unstale":"2023-11-28T11:51:17.881318+0000","last_undegraded":"2023-11-28T11:51:17.881318+0000","last_fullsized":"2023-11-28T11:51:17.881318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:39:17.843307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093581+0000","last_change":"2023-11-28T11:51:16.882735+0000","last_active":"2023-11-28T11:51:19.093581+0000","last_peered":"2023-11-28T11:51:19.093581+0000","last_clean":"2023-11-28T11:51:19.093581+0000","last_became_active":"2023-11-28T11:51:16.882478+0000","last_became_peered":"2023-11-28T11:51:16.882478+0000","last_unstale":"2023-11-28T11:51:19.093581+0000","last_undegraded":"2023-11-28T11:51:19.093581+0000","last_fullsized":"2023-11-28T11:51:19.093581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:02.988944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882570+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:51:16.882570+0000","last_peered":"2023-11-28T11:51:16.882570+0000","last_clean":"2023-11-28T11:51:16.882570+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:51:16.882570+0000","last_undegraded":"2023-11-28T11:51:16.882570+0000","last_fullsized":"2023-11-28T11:51:16.882570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093667+0000","last_change":"2023-11-28T11:51:16.483267+0000","last_active":"2023-11-28T11:51:19.093667+0000","last_peered":"2023-11-28T11:51:19.093667+0000","last_clean":"2023-11-28T11:51:19.093667+0000","last_became_active":"2023-11-28T11:51:16.482544+0000","last_became_peered":"2023-11-28T11:51:16.482544+0000","last_unstale":"2023-11-28T11:51:19.093667+0000","last_undegraded":"2023-11-28T11:51:19.093667+0000","last_fullsized":"2023-11-28T11:51:19.093667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:24:40.095304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093117+0000","last_change":"2023-11-28T11:51:16.888841+0000","last_active":"2023-11-28T11:51:19.093117+0000","last_peered":"2023-11-28T11:51:19.093117+0000","last_clean":"2023-11-28T11:51:19.093117+0000","last_became_active":"2023-11-28T11:51:16.886560+0000","last_became_peered":"2023-11-28T11:51:16.886560+0000","last_unstale":"2023-11-28T11:51:19.093117+0000","last_undegraded":"2023-11-28T11:51:19.093117+0000","last_fullsized":"2023-11-28T11:51:19.093117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:15.849625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T11:51:16.882346+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:51:16.882346+0000","last_peered":"2023-11-28T11:51:16.882346+0000","last_clean":"2023-11-28T11:51:16.882346+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:51:16.882346+0000","last_undegraded":"2023-11-28T11:51:16.882346+0000","last_fullsized":"2023-11-28T11:51:16.882346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879570+0000","last_change":"2023-11-28T11:51:16.475455+0000","last_active":"2023-11-28T11:51:17.879570+0000","last_peered":"2023-11-28T11:51:17.879570+0000","last_clean":"2023-11-28T11:51:17.879570+0000","last_became_active":"2023-11-28T11:51:16.475159+0000","last_became_peered":"2023-11-28T11:51:16.475159+0000","last_unstale":"2023-11-28T11:51:17.879570+0000","last_undegraded":"2023-11-28T11:51:17.879570+0000","last_fullsized":"2023-11-28T11:51:17.879570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:55.964379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093769+0000","last_change":"2023-11-28T11:51:16.880638+0000","last_active":"2023-11-28T11:51:19.093769+0000","last_peered":"2023-11-28T11:51:19.093769+0000","last_clean":"2023-11-28T11:51:19.093769+0000","last_became_active":"2023-11-28T11:51:16.880329+0000","last_became_peered":"2023-11-28T11:51:16.880329+0000","last_unstale":"2023-11-28T11:51:19.093769+0000","last_undegraded":"2023-11-28T11:51:19.093769+0000","last_fullsized":"2023-11-28T11:51:19.093769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:34:42.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092710+0000","last_change":"2023-11-28T11:51:16.490914+0000","last_active":"2023-11-28T11:51:19.092710+0000","last_peered":"2023-11-28T11:51:19.092710+0000","last_clean":"2023-11-28T11:51:19.092710+0000","last_became_active":"2023-11-28T11:51:16.490638+0000","last_became_peered":"2023-11-28T11:51:16.490638+0000","last_unstale":"2023-11-28T11:51:19.092710+0000","last_undegraded":"2023-11-28T11:51:19.092710+0000","last_fullsized":"2023-11-28T11:51:19.092710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:42.375209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.092793+0000","last_change":"2023-11-28T11:51:16.891450+0000","last_active":"2023-11-28T11:51:19.092793+0000","last_peered":"2023-11-28T11:51:19.092793+0000","last_clean":"2023-11-28T11:51:19.092793+0000","last_became_active":"2023-11-28T11:51:16.891183+0000","last_became_peered":"2023-11-28T11:51:16.891183+0000","last_unstale":"2023-11-28T11:51:19.092793+0000","last_undegraded":"2023-11-28T11:51:19.092793+0000","last_fullsized":"2023-11-28T11:51:19.092793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:44.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879331+0000","last_change":"2023-11-28T11:51:16.478457+0000","last_active":"2023-11-28T11:51:17.879331+0000","last_peered":"2023-11-28T11:51:17.879331+0000","last_clean":"2023-11-28T11:51:17.879331+0000","last_became_active":"2023-11-28T11:51:16.478170+0000","last_became_peered":"2023-11-28T11:51:16.478170+0000","last_unstale":"2023-11-28T11:51:17.879331+0000","last_undegraded":"2023-11-28T11:51:17.879331+0000","last_fullsized":"2023-11-28T11:51:17.879331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881448+0000","last_change":"2023-11-28T11:51:16.885759+0000","last_active":"2023-11-28T11:51:17.881448+0000","last_peered":"2023-11-28T11:51:17.881448+0000","last_clean":"2023-11-28T11:51:17.881448+0000","last_became_active":"2023-11-28T11:51:16.885461+0000","last_became_peered":"2023-11-28T11:51:16.885461+0000","last_unstale":"2023-11-28T11:51:17.881448+0000","last_undegraded":"2023-11-28T11:51:17.881448+0000","last_fullsized":"2023-11-28T11:51:17.881448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:14:57.172276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.879652+0000","last_change":"2023-11-28T11:51:16.478962+0000","last_active":"2023-11-28T11:51:17.879652+0000","last_peered":"2023-11-28T11:51:17.879652+0000","last_clean":"2023-11-28T11:51:17.879652+0000","last_became_active":"2023-11-28T11:51:16.478687+0000","last_became_peered":"2023-11-28T11:51:16.478687+0000","last_unstale":"2023-11-28T11:51:17.879652+0000","last_undegraded":"2023-11-28T11:51:17.879652+0000","last_fullsized":"2023-11-28T11:51:17.879652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:43.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881454+0000","last_change":"2023-11-28T11:51:16.891408+0000","last_active":"2023-11-28T11:51:17.881454+0000","last_peered":"2023-11-28T11:51:17.881454+0000","last_clean":"2023-11-28T11:51:17.881454+0000","last_became_active":"2023-11-28T11:51:16.891119+0000","last_became_peered":"2023-11-28T11:51:16.891119+0000","last_unstale":"2023-11-28T11:51:17.881454+0000","last_undegraded":"2023-11-28T11:51:17.881454+0000","last_fullsized":"2023-11-28T11:51:17.881454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880602+0000","last_change":"2023-11-28T11:51:16.483925+0000","last_active":"2023-11-28T11:51:17.880602+0000","last_peered":"2023-11-28T11:51:17.880602+0000","last_clean":"2023-11-28T11:51:17.880602+0000","last_became_active":"2023-11-28T11:51:16.483640+0000","last_became_peered":"2023-11-28T11:51:16.483640+0000","last_unstale":"2023-11-28T11:51:17.880602+0000","last_undegraded":"2023-11-28T11:51:17.880602+0000","last_fullsized":"2023-11-28T11:51:17.880602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:48.040083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880383+0000","last_change":"2023-11-28T11:51:16.884604+0000","last_active":"2023-11-28T11:51:17.880383+0000","last_peered":"2023-11-28T11:51:17.880383+0000","last_clean":"2023-11-28T11:51:17.880383+0000","last_became_active":"2023-11-28T11:51:16.884176+0000","last_became_peered":"2023-11-28T11:51:16.884176+0000","last_unstale":"2023-11-28T11:51:17.880383+0000","last_undegraded":"2023-11-28T11:51:17.880383+0000","last_fullsized":"2023-11-28T11:51:17.880383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:54.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093566+0000","last_change":"2023-11-28T11:51:16.488286+0000","last_active":"2023-11-28T11:51:19.093566+0000","last_peered":"2023-11-28T11:51:19.093566+0000","last_clean":"2023-11-28T11:51:19.093566+0000","last_became_active":"2023-11-28T11:51:16.487987+0000","last_became_peered":"2023-11-28T11:51:16.487987+0000","last_unstale":"2023-11-28T11:51:19.093566+0000","last_undegraded":"2023-11-28T11:51:19.093566+0000","last_fullsized":"2023-11-28T11:51:19.093566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:15.452744+0000","last_clean_scrub_stamp":"2023-11-28T11:51:15.452744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:39:02.250990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880976+0000","last_change":"2023-11-28T11:51:16.888775+0000","last_active":"2023-11-28T11:51:17.880976+0000","last_peered":"2023-11-28T11:51:17.880976+0000","last_clean":"2023-11-28T11:51:17.880976+0000","last_became_active":"2023-11-28T11:51:16.888485+0000","last_became_peered":"2023-11-28T11:51:16.888485+0000","last_unstale":"2023-11-28T11:51:17.880976+0000","last_undegraded":"2023-11-28T11:51:17.880976+0000","last_fullsized":"2023-11-28T11:51:17.880976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:55.254375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093487+0000","last_change":"2023-11-28T11:51:16.888856+0000","last_active":"2023-11-28T11:51:19.093487+0000","last_peered":"2023-11-28T11:51:19.093487+0000","last_clean":"2023-11-28T11:51:19.093487+0000","last_became_active":"2023-11-28T11:51:16.887628+0000","last_became_peered":"2023-11-28T11:51:16.887628+0000","last_unstale":"2023-11-28T11:51:19.093487+0000","last_undegraded":"2023-11-28T11:51:19.093487+0000","last_fullsized":"2023-11-28T11:51:19.093487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880202+0000","last_change":"2023-11-28T11:51:16.887897+0000","last_active":"2023-11-28T11:51:17.880202+0000","last_peered":"2023-11-28T11:51:17.880202+0000","last_clean":"2023-11-28T11:51:17.880202+0000","last_became_active":"2023-11-28T11:51:16.887663+0000","last_became_peered":"2023-11-28T11:51:16.887663+0000","last_unstale":"2023-11-28T11:51:17.880202+0000","last_undegraded":"2023-11-28T11:51:17.880202+0000","last_fullsized":"2023-11-28T11:51:17.880202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:07.282120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.881169+0000","last_change":"2023-11-28T11:51:16.891805+0000","last_active":"2023-11-28T11:51:17.881169+0000","last_peered":"2023-11-28T11:51:17.881169+0000","last_clean":"2023-11-28T11:51:17.881169+0000","last_became_active":"2023-11-28T11:51:16.891091+0000","last_became_peered":"2023-11-28T11:51:16.891091+0000","last_unstale":"2023-11-28T11:51:17.881169+0000","last_undegraded":"2023-11-28T11:51:17.881169+0000","last_fullsized":"2023-11-28T11:51:17.881169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:52.863792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:17.880475+0000","last_change":"2023-11-28T11:51:16.897807+0000","last_active":"2023-11-28T11:51:17.880475+0000","last_peered":"2023-11-28T11:51:17.880475+0000","last_clean":"2023-11-28T11:51:17.880475+0000","last_became_active":"2023-11-28T11:51:16.897481+0000","last_became_peered":"2023-11-28T11:51:16.897481+0000","last_unstale":"2023-11-28T11:51:17.880475+0000","last_undegraded":"2023-11-28T11:51:17.880475+0000","last_fullsized":"2023-11-28T11:51:17.880475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:28.221251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T11:51:19.093497+0000","last_change":"2023-11-28T11:51:16.886025+0000","last_active":"2023-11-28T11:51:19.093497+0000","last_peered":"2023-11-28T11:51:19.093497+0000","last_clean":"2023-11-28T11:51:19.093497+0000","last_became_active":"2023-11-28T11:51:16.885791+0000","last_became_peered":"2023-11-28T11:51:16.885791+0000","last_unstale":"2023-11-28T11:51:19.093497+0000","last_undegraded":"2023-11-28T11:51:19.093497+0000","last_fullsized":"2023-11-28T11:51:19.093497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:16.460227+0000","last_clean_scrub_stamp":"2023-11-28T11:51:16.460227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:47:09.888720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9092,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362748,"statfs":{"total":96636764160,"available":96627453952,"internally_reserved":0,"allocated":847872,"data_stored":674887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9064,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94362776,"statfs":{"total":96636764160,"available":96627482624,"internally_reserved":0,"allocated":819200,"data_stored":673123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8036,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363804,"statfs":{"total":96636764160,"available":96628535296,"internally_reserved":0,"allocated":225280,"data_stored":82335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7908,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363932,"statfs":{"total":96636764160,"available":96628666368,"internally_reserved":0,"allocated":225280,"data_stored":82755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8040,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363800,"statfs":{"total":96636764160,"available":96628531200,"internally_reserved":0,"allocated":229376,"data_stored":82845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8056,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363784,"statfs":{"total":96636764160,"available":96628514816,"internally_reserved":0,"allocated":245760,"data_stored":84359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7976,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":7744,"kb_avail":94363864,"statfs":{"total":96636764160,"available":96628596736,"internally_reserved":0,"allocated":229376,"data_stored":82405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7929856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:51:25.921 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:51:25.921 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T11:51:25.921 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T11:51:25.922 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:51:26.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.072+0000 7f7e7f7ae700 1 Processor -- start 2023-11-28T11:51:26.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.072+0000 7f7e7f7ae700 1 -- start start 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7f7ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e780b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7f7ae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7e780b2e70 0x7f7e7806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7f7ae700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7813ff00 con 0x7f7e780cd8c0 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7f7ae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e781404c0 con 0x7f7e780b2e70 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7f7ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78140dc0 con 0x7f7e78140950 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7cd49700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e780b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7dd4b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7e780b2e70 0x7f7e7806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7cd49700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e780b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:46970/0 (socket says 172.21.15.144:46970) 2023-11-28T11:51:26.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7cd49700 1 -- 172.21.15.144:0/847308446 learned_addr learned my addr 172.21.15.144:0/847308446 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:26.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7dd4b700 1 -- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 msgr2=0x7f7e780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:26.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7dd4b700 1 --2- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e780b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7dd4b700 1 -- 172.21.15.144:0/847308446 >> v1:172.21.15.177:6789/0 conn(0x7f7e780cd8c0 legacy=0x7f7e780cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7dd4b700 1 -- 172.21.15.144:0/847308446 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7815aa80 con 0x7f7e780b2e70 2023-11-28T11:51:26.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e7dd4b700 1 --2- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e7806f850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009d90 tx=0x7f7e7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ab07ab505e0a97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.073+0000 7f7e6ffff700 1 -- 172.21.15.144:0/847308446 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000e070 con 0x7f7e780b2e70 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e6ffff700 1 -- 172.21.15.144:0/847308446 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7000bb20 con 0x7f7e780b2e70 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e6ffff700 1 -- 172.21.15.144:0/847308446 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70012430 con 0x7f7e780b2e70 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 msgr2=0x7f7e7806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e7806f850 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009d90 tx=0x7f7e7000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/847308446 shutdown_connections 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e7806f850 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/847308446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e780b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/847308446 >> 172.21.15.144:0/847308446 conn(0x7f7e7806aed0 msgr2=0x7f7e7806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:26.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/847308446 shutdown_connections 2023-11-28T11:51:26.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.074+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/847308446 wait complete. 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 Processor -- start 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 -- start start 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e780c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 0x7f7e780b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e78143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7806bc90 con 0x7f7e780cd8c0 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78143eb0 con 0x7f7e780b2e70 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7f7ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7806b380 con 0x7f7e78140950 2023-11-28T11:51:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.076+0000 7f7e7cd49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 0x7f7e780b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 0x7f7e780b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34894/0 (socket says 172.21.15.144:34894) 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 -- 172.21.15.144:0/1728232520 learned_addr learned my addr 172.21.15.144:0/1728232520 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7d54a700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e780c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7dd4b700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e78143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 -- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 msgr2=0x7f7e78143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e78143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 -- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 msgr2=0x7f7e780c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e780c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 -- 172.21.15.144:0/1728232520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009a70 con 0x7f7e780cd8c0 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7cd49700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 0x7f7e780b2370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800e800 tx=0x7f7e6800eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68013680 con 0x7f7e780cd8c0 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e680137e0 con 0x7f7e780cd8c0 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e780b35f0 con 0x7f7e780cd8c0 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68013b70 con 0x7f7e780cd8c0 2023-11-28T11:51:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.077+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e780b3310 con 0x7f7e780cd8c0 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.078+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e78140150 con 0x7f7e780cd8c0 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.079+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e68019030 con 0x7f7e780cd8c0 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.079+0000 7f7e6dffb700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7e5c068320 0x7f7e5c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.079+0000 7f7e7d54a700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7e5c068320 0x7f7e5c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.079+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e680858e0 con 0x7f7e780cd8c0 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.080+0000 7f7e7d54a700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7e5c068320 0x7f7e5c06a7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000fb0 tx=0x7f7e74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.083+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7e68052140 con 0x7f7e780cd8c0 2023-11-28T11:51:26.312 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.311+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7e7813fcf0 con 0x7f7e780cd8c0 2023-11-28T11:51:26.312 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.311+0000 7f7e6dffb700 1 -- 172.21.15.144:0/1728232520 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7e68017040 con 0x7f7e780cd8c0 2023-11-28T11:51:26.312 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:26.312 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T11:51:26.314 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7e5c068320 msgr2=0x7f7e5c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7e5c068320 0x7f7e5c06a7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000fb0 tx=0x7f7e74009040 comp rx=0 tx=0).stop 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 msgr2=0x7f7e780b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 0x7f7e780b2370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800e800 tx=0x7f7e6800eb10 comp rx=0 tx=0).stop 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 shutdown_connections 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7e5c068320 0x7f7e5c06a7d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e78140950 0x7f7e78143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e780b2e70 0x7f7e780c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 --2- 172.21.15.144:0/1728232520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e780cd8c0 0x7f7e780b2370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 >> 172.21.15.144:0/1728232520 conn(0x7f7e7806aed0 msgr2=0x7f7e780cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 shutdown_connections 2023-11-28T11:51:26.315 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.313+0000 7f7e7f7ae700 1 -- 172.21.15.144:0/1728232520 wait complete. 2023-11-28T11:51:26.326 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T11:51:26.327 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:26.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.445+0000 7fd190652700 1 Processor -- start 2023-11-28T11:51:26.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.445+0000 7fd190652700 1 -- start start 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd190652700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd18806dee0 0x7fd18806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd190652700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18815d060 con 0x7fd188160540 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd190652700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18815d340 con 0x7fd188071040 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd190652700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815d620 con 0x7fd18806dee0 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18ebef700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd188160540 0x7fd1881609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51616/0 (socket says 172.21.15.144:51616) 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18ebef700 1 -- 172.21.15.144:0/84388782 learned_addr learned my addr 172.21.15.144:0/84388782 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18dbed700 1 --2- 172.21.15.144:0/84388782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd18806dee0 0x7fd18806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259176404 0 0) 0x7fd18815d060 con 0x7fd188160540 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1740035b0 con 0x7fd188160540 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269429546 0 0) 0x7fd18815d340 con 0x7fd188071040 2023-11-28T11:51:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd18815d060 con 0x7fd188071040 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18dbed700 1 -- 172.21.15.144:0/84388782 >> v1:172.21.15.177:6789/0 conn(0x7fd188071040 legacy=0x7fd188071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18dbed700 1 -- 172.21.15.144:0/84388782 >> v1:172.21.15.144:6789/0 conn(0x7fd188160540 legacy=0x7fd1881609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18dbed700 1 -- 172.21.15.144:0/84388782 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18815d900 con 0x7fd18806dee0 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.446+0000 7fd18dbed700 1 --2- 172.21.15.144:0/84388782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd18806dee0 0x7fd18806e330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd178009870 tx=0x7fd178009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6026b50519d38d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178011830 con 0x7fd18806dee0 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178011990 con 0x7fd18806dee0 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd18d3ec700 1 -- 172.21.15.144:0/84388782 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178011d20 con 0x7fd18806dee0 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 -- 172.21.15.144:0/84388782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd18806dee0 msgr2=0x7fd18806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 --2- 172.21.15.144:0/84388782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd18806dee0 0x7fd18806e330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd178009870 tx=0x7fd178009b80 comp rx=0 tx=0).stop 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 -- 172.21.15.144:0/84388782 shutdown_connections 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 --2- 172.21.15.144:0/84388782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd18806dee0 0x7fd18806e330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 -- 172.21.15.144:0/84388782 >> 172.21.15.144:0/84388782 conn(0x7fd1880d0000 msgr2=0x7fd1880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 -- 172.21.15.144:0/84388782 shutdown_connections 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.447+0000 7fd190652700 1 -- 172.21.15.144:0/84388782 wait complete. 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 Processor -- start 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 -- start start 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd18806dee0 0x7fd1880b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd188071040 0x7fd1880c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 0x7fd1880c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b3ff0 con 0x7fd188160540 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b3090 con 0x7fd18806dee0 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd190652700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b3370 con 0x7fd188071040 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd18ebef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 0x7fd1880c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd18ebef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 0x7fd1880c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34910/0 (socket says 172.21.15.144:34910) 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd18ebef700 1 -- 172.21.15.144:0/4273250306 learned_addr learned my addr 172.21.15.144:0/4273250306 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd18dbed700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd188071040 0x7fd1880c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.449+0000 7fd18e3ee700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd18806dee0 0x7fd1880b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd18ebef700 1 -- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd188071040 msgr2=0x7fd1880c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd18ebef700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd188071040 0x7fd1880c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd18ebef700 1 -- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd18806dee0 msgr2=0x7fd1880b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd18ebef700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd18806dee0 0x7fd1880b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd18ebef700 1 -- 172.21.15.144:0/4273250306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780094a0 con 0x7fd188160540 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd18ebef700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 0x7fd1880c4f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd18000caa0 tx=0x7fd18000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000a9e0 con 0x7fd188160540 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000ab60 con 0x7fd188160540 2023-11-28T11:51:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18815ccd0 con 0x7fd188160540 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180012400 con 0x7fd188160540 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.450+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18806e870 con 0x7fd188160540 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.451+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18806a270 con 0x7fd188160540 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.451+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd180019030 con 0x7fd188160540 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.452+0000 7fd17effd700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd16c0682f0 0x7fd16c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.452+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd180085720 con 0x7fd188160540 2023-11-28T11:51:26.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.452+0000 7fd18e3ee700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd16c0682f0 0x7fd16c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.452+0000 7fd18e3ee700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd16c0682f0 0x7fd16c06a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1840011e0 tx=0x7fd184009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.456+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd180052040 con 0x7fd188160540 2023-11-28T11:51:26.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.641+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1880cef50 con 0x7fd188160540 2023-11-28T11:51:26.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.642+0000 7fd17effd700 1 -- 172.21.15.144:0/4273250306 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fd180055660 con 0x7fd188160540 2023-11-28T11:51:26.644 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:26.644 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd16c0682f0 msgr2=0x7fd16c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd16c0682f0 0x7fd16c06a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1840011e0 tx=0x7fd184009040 comp rx=0 tx=0).stop 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 msgr2=0x7fd1880c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 0x7fd1880c4f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd18000caa0 tx=0x7fd18000cdb0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 shutdown_connections 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd188160540 0x7fd1880c4f20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd16c0682f0 0x7fd16c06a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd18806dee0 0x7fd1880b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 --2- 172.21.15.144:0/4273250306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd188071040 0x7fd1880c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 >> 172.21.15.144:0/4273250306 conn(0x7fd1880d0000 msgr2=0x7fd18815e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:26.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 shutdown_connections 2023-11-28T11:51:26.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.645+0000 7fd190652700 1 -- 172.21.15.144:0/4273250306 wait complete. 2023-11-28T11:51:26.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:26.659 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:26.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.780+0000 7fa993ae7700 1 Processor -- start 2023-11-28T11:51:26.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa993ae7700 1 -- start start 2023-11-28T11:51:26.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa993ae7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa98c1658c0 0x7fa98c167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa993ae7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c15d060 con 0x7fa98c071040 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa993ae7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c15d340 con 0x7fa98c06dee0 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa993ae7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c161550 con 0x7fa98c1658c0 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa992084700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa98c1658c0 0x7fa98c167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.781+0000 7fa991883700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa98c071040 0x7fa98c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51626/0 (socket says 172.21.15.144:51626) 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa991883700 1 -- 172.21.15.144:0/509187132 learned_addr learned my addr 172.21.15.144:0/509187132 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa992084700 1 -- 172.21.15.144:0/509187132 >> v1:172.21.15.177:6789/0 conn(0x7fa98c06dee0 legacy=0x7fa98c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa992084700 1 -- 172.21.15.144:0/509187132 >> v1:172.21.15.144:6789/0 conn(0x7fa98c071040 legacy=0x7fa98c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa992084700 1 -- 172.21.15.144:0/509187132 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c15d620 con 0x7fa98c1658c0 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa992084700 1 --2- 172.21.15.144:0/509187132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c1658c0 0x7fa98c167ca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa984009d90 tx=0x7fa98400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4d701e6cc4d5bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa990881700 1 -- 172.21.15.144:0/509187132 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98400e070 con 0x7fa98c1658c0 2023-11-28T11:51:26.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa990881700 1 -- 172.21.15.144:0/509187132 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98400bbd0 con 0x7fa98c1658c0 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.782+0000 7fa990881700 1 -- 172.21.15.144:0/509187132 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984012430 con 0x7fa98c1658c0 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 -- 172.21.15.144:0/509187132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c1658c0 msgr2=0x7fa98c167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 --2- 172.21.15.144:0/509187132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c1658c0 0x7fa98c167ca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa984009d90 tx=0x7fa98400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 -- 172.21.15.144:0/509187132 shutdown_connections 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 --2- 172.21.15.144:0/509187132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c1658c0 0x7fa98c167ca0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 -- 172.21.15.144:0/509187132 >> 172.21.15.144:0/509187132 conn(0x7fa98c0d0000 msgr2=0x7fa98c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 -- 172.21.15.144:0/509187132 shutdown_connections 2023-11-28T11:51:26.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.783+0000 7fa993ae7700 1 -- 172.21.15.144:0/509187132 wait complete. 2023-11-28T11:51:26.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.784+0000 7fa993ae7700 1 Processor -- start 2023-11-28T11:51:26.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.784+0000 7fa993ae7700 1 -- start start 2023-11-28T11:51:26.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 0x7fa98c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa98c071040 0x7fa98c149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa98c1658c0 0x7fa98c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c149a00 con 0x7fa98c06dee0 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c5770 con 0x7fa98c1658c0 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c4b60 con 0x7fa98c071040 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 0x7fa98c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 0x7fa98c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34920/0 (socket says 172.21.15.144:34920) 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 -- 172.21.15.144:0/2527200830 learned_addr learned my addr 172.21.15.144:0/2527200830 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa992084700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa98c1658c0 0x7fa98c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991082700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa98c071040 0x7fa98c149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 -- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa98c071040 msgr2=0x7fa98c149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa98c071040 0x7fa98c149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 -- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa98c1658c0 msgr2=0x7fa98c1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa98c1658c0 0x7fa98c1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 -- 172.21.15.144:0/2527200830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009a70 con 0x7fa98c06dee0 2023-11-28T11:51:26.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991082700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa98c071040 0x7fa98c149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:26.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa992084700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa98c1658c0 0x7fa98c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa991883700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 0x7fa98c0b2e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00c480 tx=0x7fa97c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c005d70 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c0070d0 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c15ccd0 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c007440 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.785+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c15d900 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.786+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c06a270 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.787+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa97c019030 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.787+0000 7fa9827fc700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa9700682f0 0x7fa97006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.787+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa97c084f60 con 0x7fa98c06dee0 2023-11-28T11:51:26.791 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.787+0000 7fa991082700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa9700682f0 0x7fa97006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:26.792 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.787+0000 7fa991082700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa9700682f0 0x7fa97006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0af4a0 tx=0x7fa988009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:26.792 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.791+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa97c04b5e0 con 0x7fa98c06dee0 2023-11-28T11:51:26.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.974+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa98c0720a0 con 0x7fa98c06dee0 2023-11-28T11:51:26.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.974+0000 7fa9827fc700 1 -- 172.21.15.144:0/2527200830 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa97c0170a0 con 0x7fa98c06dee0 2023-11-28T11:51:26.975 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:26.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa9700682f0 msgr2=0x7fa97006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa9700682f0 0x7fa97006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0af4a0 tx=0x7fa988009040 comp rx=0 tx=0).stop 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 msgr2=0x7fa98c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 0x7fa98c0b2e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00c480 tx=0x7fa97c00c840 comp rx=0 tx=0).stop 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 shutdown_connections 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c06dee0 0x7fa98c0b2e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa9700682f0 0x7fa97006a7a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa98c1658c0 0x7fa98c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 --2- 172.21.15.144:0/2527200830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa98c071040 0x7fa98c149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 >> 172.21.15.144:0/2527200830 conn(0x7fa98c0d0000 msgr2=0x7fa98c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 shutdown_connections 2023-11-28T11:51:26.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:26.976+0000 7fa993ae7700 1 -- 172.21.15.144:0/2527200830 wait complete. 2023-11-28T11:51:26.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:27.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.112+0000 7f0160f20700 1 Processor -- start 2023-11-28T11:51:27.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.112+0000 7f0160f20700 1 -- start start 2023-11-28T11:51:27.116 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0160f20700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 0x7f015c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0160f20700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f015c06ba90 0x7f015c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0160f20700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0cc5f0 con 0x7f015c0cbdf0 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0160f20700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c15d470 con 0x7f015c06ba90 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0160f20700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15d750 con 0x7f015c15cef0 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0159d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f015c0cbdf0 0x7f015c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51640/0 (socket says 172.21.15.144:51640) 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f0159d9b700 1 -- 172.21.15.144:0/2768174056 learned_addr learned my addr 172.21.15.144:0/2768174056 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015a59c700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 0x7f015c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2768174056 >> v2:172.21.15.177:3300/0 conn(0x7f015c06ba90 0x7f015c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653987918 0 0) 0x7f015c0cc5f0 con 0x7f015c0cbdf0 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01440035b0 con 0x7f015c0cbdf0 2023-11-28T11:51:27.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911943773 0 0) 0x7f01440035b0 con 0x7f015c0cbdf0 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 msgr2=0x7f015c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 0x7f015c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 >> v2:172.21.15.177:3300/0 conn(0x7f015c06ba90 msgr2=0x7f015c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 --2- 172.21.15.144:0/2768174056 >> v2:172.21.15.177:3300/0 conn(0x7f015c06ba90 0x7f015c0ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c15da30 con 0x7f015c0cbdf0 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f0150001670 con 0x7f015c0cbdf0 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2768174056 >> v2:172.21.15.177:3300/0 conn(0x7f015c06ba90 0x7f015c0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 >> v1:172.21.15.144:6789/0 conn(0x7f015c0cbdf0 legacy=0x7f015c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144003a00 0x7f0144003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 0x7f0144004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0144004d90 0x7f0144005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15d470 con 0x7f0144004d90 2023-11-28T11:51:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c15d750 con 0x7f0144003a00 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.113+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014400ba10 con 0x7f015c15cef0 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0144004d90 0x7f0144005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015a59c700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144003a00 0x7f0144003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f0159d9b700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 0x7f0144004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 -- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 msgr2=0x7f0144004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c15cef0 0x7f0144004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 -- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144003a00 msgr2=0x7f0144003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144003a00 0x7f0144003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 -- 172.21.15.144:0/2768174056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440035b0 con 0x7f0144004d90 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0144004d90 0x7f0144005200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f015400e350 tx=0x7f0144005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe49bb673c531af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400ed90 con 0x7f0144004d90 2023-11-28T11:51:27.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015400f040 con 0x7f0144004d90 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.114+0000 7f015959a700 1 -- 172.21.15.144:0/2768174056 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540146a0 con 0x7f0144004d90 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f0160f20700 1 -- 172.21.15.144:0/2768174056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0144004d90 msgr2=0x7f0144005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f0160f20700 1 --2- 172.21.15.144:0/2768174056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0144004d90 0x7f0144005200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f015400e350 tx=0x7f0144005880 comp rx=0 tx=0).stop 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f015ad9d700 1 -- 172.21.15.144:0/2768174056 reap_dead start 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f0160f20700 1 -- 172.21.15.144:0/2768174056 shutdown_connections 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f0160f20700 1 -- 172.21.15.144:0/2768174056 >> 172.21.15.144:0/2768174056 conn(0x7f015c0d0000 msgr2=0x7f015c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f0160f20700 1 -- 172.21.15.144:0/2768174056 shutdown_connections 2023-11-28T11:51:27.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.115+0000 7f0160f20700 1 -- 172.21.15.144:0/2768174056 wait complete. 2023-11-28T11:51:27.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.116+0000 7f0160f20700 1 Processor -- start 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.116+0000 7f0160f20700 1 -- start start 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144004d90 0x7f015c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c0c70d0 0x7f015c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 0x7f015c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c155700 con 0x7f015c0b56d0 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c0b5be0 con 0x7f0144004d90 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c156a50 con 0x7f015c0c70d0 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 0x7f015c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 0x7f015c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34944/0 (socket says 172.21.15.144:34944) 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 -- 172.21.15.144:0/2451851057 learned_addr learned my addr 172.21.15.144:0/2451851057 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0159d9b700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c0c70d0 0x7f015c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 -- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c0c70d0 msgr2=0x7f015c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c0c70d0 0x7f015c0b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 -- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144004d90 msgr2=0x7f015c1551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144004d90 0x7f015c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 -- 172.21.15.144:0/2451851057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c15dff0 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f015ad9d700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 0x7f015c15a5c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f01440046d0 tx=0x7f0154009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154007000 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154009410 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c15dd10 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540276b0 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.117+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c0af400 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.118+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0d0e60 con 0x7f015c0b56d0 2023-11-28T11:51:27.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.119+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f015401e430 con 0x7f015c0b56d0 2023-11-28T11:51:27.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.119+0000 7f014affd700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f013c0682f0 0x7f013c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.119+0000 7f015a59c700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f013c0682f0 0x7f013c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.119+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0154087f40 con 0x7f015c0b56d0 2023-11-28T11:51:27.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.119+0000 7f015a59c700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f013c0682f0 0x7f013c06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f014c009ae0 tx=0x7f014c0093e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.123+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0154054740 con 0x7f015c0b56d0 2023-11-28T11:51:27.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.303+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f015c156b90 con 0x7f015c0b56d0 2023-11-28T11:51:27.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.303+0000 7f014affd700 1 -- 172.21.15.144:0/2451851057 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f015c156b90 con 0x7f015c0b56d0 2023-11-28T11:51:27.304 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f013c0682f0 msgr2=0x7f013c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f013c0682f0 0x7f013c06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f014c009ae0 tx=0x7f014c0093e0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 msgr2=0x7f015c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 0x7f015c15a5c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f01440046d0 tx=0x7f0154009750 comp rx=0 tx=0).stop 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 shutdown_connections 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f015c0b56d0 0x7f015c15a5c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f013c0682f0 0x7f013c06a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144004d90 0x7f015c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 --2- 172.21.15.144:0/2451851057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f015c0c70d0 0x7f015c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 >> 172.21.15.144:0/2451851057 conn(0x7f015c0d0000 msgr2=0x7f015c0c66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:27.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 shutdown_connections 2023-11-28T11:51:27.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.306+0000 7f0160f20700 1 -- 172.21.15.144:0/2451851057 wait complete. 2023-11-28T11:51:27.319 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:27.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 Processor -- start 2023-11-28T11:51:27.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 -- start start 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc87c0b20b0 0x7fc87c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc87c0b29b0 0x7fc87c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c15cec0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c0b0230 con 0x7fc87c0b29b0 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.441+0000 7fc882e5c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b4570 con 0x7fc87c0d01b0 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc87c0b20b0 0x7fc87c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc87c0b20b0 0x7fc87c0d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34952/0 (socket says 172.21.15.144:34952) 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 -- 172.21.15.144:0/2219743004 learned_addr learned my addr 172.21.15.144:0/2219743004 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc880bf8700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc8813f9700 1 --2- 172.21.15.144:0/2219743004 >> v2:172.21.15.177:3300/0 conn(0x7fc87c0b29b0 0x7fc87c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 -- 172.21.15.144:0/2219743004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 msgr2=0x7fc87c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 -- 172.21.15.144:0/2219743004 >> v2:172.21.15.177:3300/0 conn(0x7fc87c0b29b0 msgr2=0x7fc87c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 --2- 172.21.15.144:0/2219743004 >> v2:172.21.15.177:3300/0 conn(0x7fc87c0b29b0 0x7fc87c0b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 -- 172.21.15.144:0/2219743004 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c0b4850 con 0x7fc87c0b20b0 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc880bf8700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:27.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc8813f9700 1 --2- 172.21.15.144:0/2219743004 >> v2:172.21.15.177:3300/0 conn(0x7fc87c0b29b0 0x7fc87c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:27.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87bfff700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c0d1160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc87c066e80 tx=0x7fc868009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967990f53453ed5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87b7fe700 1 -- 172.21.15.144:0/2219743004 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868011830 con 0x7fc87c0b20b0 2023-11-28T11:51:27.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87b7fe700 1 -- 172.21.15.144:0/2219743004 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc868011990 con 0x7fc87c0b20b0 2023-11-28T11:51:27.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.442+0000 7fc87b7fe700 1 -- 172.21.15.144:0/2219743004 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868011d20 con 0x7fc87c0b20b0 2023-11-28T11:51:27.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 -- 172.21.15.144:0/2219743004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 msgr2=0x7fc87c0d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c0d1160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc87c066e80 tx=0x7fc868009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 -- 172.21.15.144:0/2219743004 shutdown_connections 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 --2- 172.21.15.144:0/2219743004 >> v2:172.21.15.177:3300/0 conn(0x7fc87c0b29b0 0x7fc87c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 --2- 172.21.15.144:0/2219743004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c0d1160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 -- 172.21.15.144:0/2219743004 >> 172.21.15.144:0/2219743004 conn(0x7fc87c09c3f0 msgr2=0x7fc87c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 -- 172.21.15.144:0/2219743004 shutdown_connections 2023-11-28T11:51:27.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.443+0000 7fc882e5c700 1 -- 172.21.15.144:0/2219743004 wait complete. 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.444+0000 7fc882e5c700 1 Processor -- start 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.444+0000 7fc882e5c700 1 -- start start 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc882e5c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc882e5c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc87c0b29b0 0x7fc87c0ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc882e5c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc882e5c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c1f54f0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc882e5c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0cc670 con 0x7fc87c0b29b0 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc882e5c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b8b40 con 0x7fc87c0d01b0 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34954/0 (socket says 172.21.15.144:34954) 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 -- 172.21.15.144:0/3088841001 learned_addr learned my addr 172.21.15.144:0/3088841001 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc8813f9700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc87bfff700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc87c0b29b0 0x7fc87c0ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 -- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 msgr2=0x7fc87c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 -- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc87c0b29b0 msgr2=0x7fc87c0ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc87c0b29b0 0x7fc87c0ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 -- 172.21.15.144:0/3088841001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874009a70 con 0x7fc87c0b20b0 2023-11-28T11:51:27.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc8813f9700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.445+0000 7fc880bf8700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c158c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc87000ebe0 tx=0x7fc870009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.446+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000c8b0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.446+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000ca10 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.446+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8700094a0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.446+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000cda0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.446+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc868009550 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.447+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c0afec0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.447+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc870019030 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.447+0000 7fc8797fa700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc8600682f0 0x7fc86006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.447+0000 7fc87bfff700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc8600682f0 0x7fc86006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.447+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc870085d10 con 0x7fc87c0b20b0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.448+0000 7fc87bfff700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc8600682f0 0x7fc86006a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc868009c10 tx=0x7fc868008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.452+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8700524f0 con 0x7fc87c0b20b0 2023-11-28T11:51:27.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.633+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc87c1f6030 con 0x7fc87c0b20b0 2023-11-28T11:51:27.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.634+0000 7fc8797fa700 1 -- 172.21.15.144:0/3088841001 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc870055b10 con 0x7fc87c0b20b0 2023-11-28T11:51:27.635 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:27.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc8600682f0 msgr2=0x7fc86006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc8600682f0 0x7fc86006a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc868009c10 tx=0x7fc868008030 comp rx=0 tx=0).stop 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 msgr2=0x7fc87c158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c158c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc87000ebe0 tx=0x7fc870009e20 comp rx=0 tx=0).stop 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 shutdown_connections 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc87c0b20b0 0x7fc87c158c60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc8600682f0 0x7fc86006a7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc87c0d01b0 0x7fc87c0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 --2- 172.21.15.144:0/3088841001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc87c0b29b0 0x7fc87c0ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 >> 172.21.15.144:0/3088841001 conn(0x7fc87c09c3f0 msgr2=0x7fc87c0cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 shutdown_connections 2023-11-28T11:51:27.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.636+0000 7fc882e5c700 1 -- 172.21.15.144:0/3088841001 wait complete. 2023-11-28T11:51:27.648 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:27.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.770+0000 7f7fc7f00700 1 Processor -- start 2023-11-28T11:51:27.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.771+0000 7f7fc7f00700 1 -- start start 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc7f00700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00d0a00 0x7f7fc015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc7f00700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7fc00b2930 0x7f7fc00b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc7f00700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc006b420 con 0x7f7fc00ce040 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc7f00700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc00b32f0 con 0x7f7fc00b2930 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc7f00700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc006ff60 con 0x7f7fc00d0a00 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc649d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7fc00b2930 0x7f7fc00b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc5c9c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7fc00ce040 0x7f7fc006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51674/0 (socket says 172.21.15.144:51674) 2023-11-28T11:51:27.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc5c9c700 1 -- 172.21.15.144:0/3526047317 learned_addr learned my addr 172.21.15.144:0/3526047317 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc549b700 1 --2- 172.21.15.144:0/3526047317 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00d0a00 0x7f7fc015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc649d700 1 -- 172.21.15.144:0/3526047317 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00d0a00 msgr2=0x7f7fc015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc649d700 1 --2- 172.21.15.144:0/3526047317 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00d0a00 0x7f7fc015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc649d700 1 -- 172.21.15.144:0/3526047317 >> v1:172.21.15.144:6789/0 conn(0x7f7fc00ce040 legacy=0x7f7fc006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc649d700 1 -- 172.21.15.144:0/3526047317 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc006b700 con 0x7f7fc00b2930 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc4c9a700 1 -- 172.21.15.144:0/3526047317 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527217617 0 0) 0x7f7fc006b420 con 0x7f7fc00ce040 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc649d700 1 --2- 172.21.15.144:0/3526047317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00b2930 0x7f7fc00b2db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009d90 tx=0x7f7fb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f785bfd9026b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc4c9a700 1 -- 172.21.15.144:0/3526047317 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800e070 con 0x7f7fc00b2930 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc4c9a700 1 -- 172.21.15.144:0/3526047317 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb800bbd0 con 0x7f7fc00b2930 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.772+0000 7f7fc4c9a700 1 -- 172.21.15.144:0/3526047317 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8012430 con 0x7f7fc00b2930 2023-11-28T11:51:27.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 -- 172.21.15.144:0/3526047317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00b2930 msgr2=0x7f7fc00b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/3526047317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00b2930 0x7f7fc00b2db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009d90 tx=0x7f7fb800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:27.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 -- 172.21.15.144:0/3526047317 shutdown_connections 2023-11-28T11:51:27.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/3526047317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00b2930 0x7f7fc00b2db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/3526047317 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00d0a00 0x7f7fc015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 -- 172.21.15.144:0/3526047317 >> 172.21.15.144:0/3526047317 conn(0x7f7fc006aed0 msgr2=0x7f7fc006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:27.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 -- 172.21.15.144:0/3526047317 shutdown_connections 2023-11-28T11:51:27.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.773+0000 7f7fc7f00700 1 -- 172.21.15.144:0/3526047317 wait complete. 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.774+0000 7f7fc7f00700 1 Processor -- start 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 -- start start 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00b2930 0x7f7fc00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 0x7f7fc00c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc00d0a00 0x7f7fc00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00c4f00 con 0x7f7fc00ce040 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00c51a0 con 0x7f7fc00d0a00 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.775+0000 7f7fc7f00700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00c5480 con 0x7f7fc00b2930 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 0x7f7fc00c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 0x7f7fc00c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34970/0 (socket says 172.21.15.144:34970) 2023-11-28T11:51:27.780 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 -- 172.21.15.144:0/2178184451 learned_addr learned my addr 172.21.15.144:0/2178184451 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc5c9c700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00b2930 0x7f7fc00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc649d700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc00d0a00 0x7f7fc00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 -- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00b2930 msgr2=0x7f7fc00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00b2930 0x7f7fc00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 -- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc00d0a00 msgr2=0x7f7fc00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc00d0a00 0x7f7fc00cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 -- 172.21.15.144:0/2178184451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009a70 con 0x7f7fc00ce040 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc649d700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc00d0a00 0x7f7fc00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc549b700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 0x7f7fc00c49c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c7f0 tx=0x7f7fbc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc011800 con 0x7f7fc00ce040 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc011960 con 0x7f7fc00ce040 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc006bcc0 con 0x7f7fc00ce040 2023-11-28T11:51:27.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc011cf0 con 0x7f7fc00ce040 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.776+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc006b9e0 con 0x7f7fc00ce040 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.777+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc006bfa0 con 0x7f7fc00ce040 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.778+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7fbc021070 con 0x7f7fc00ce040 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.778+0000 7f7fb67fc700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7fa40682f0 0x7f7fa406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.778+0000 7f7fc5c9c700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7fa40682f0 0x7f7fa406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.778+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fbc085a10 con 0x7f7fc00ce040 2023-11-28T11:51:27.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.779+0000 7f7fc5c9c700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7fa40682f0 0x7f7fa406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fc015e2a0 tx=0x7f7fb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:27.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.782+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7fbc052270 con 0x7f7fc00ce040 2023-11-28T11:51:27.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.965+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fc00cb270 con 0x7f7fc00ce040 2023-11-28T11:51:27.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.965+0000 7f7fb67fc700 1 -- 172.21.15.144:0/2178184451 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fbc055890 con 0x7f7fc00ce040 2023-11-28T11:51:27.966 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:27.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.967+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7fa40682f0 msgr2=0x7f7fa406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.967+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7fa40682f0 0x7f7fa406a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fc015e2a0 tx=0x7f7fb0009040 comp rx=0 tx=0).stop 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.967+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 msgr2=0x7f7fc00c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.967+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 0x7f7fc00c49c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c7f0 tx=0x7f7fbc00cbb0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 shutdown_connections 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7fa40682f0 0x7f7fa406a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fc00d0a00 0x7f7fc00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fc00b2930 0x7f7fc00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 --2- 172.21.15.144:0/2178184451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc00ce040 0x7f7fc00c49c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 >> 172.21.15.144:0/2178184451 conn(0x7f7fc006aed0 msgr2=0x7f7fc015ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 shutdown_connections 2023-11-28T11:51:27.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:27.968+0000 7f7fc7f00700 1 -- 172.21.15.144:0/2178184451 wait complete. 2023-11-28T11:51:27.981 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.100+0000 7f56d661a700 1 Processor -- start 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.100+0000 7f56d661a700 1 -- start start 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56d661a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f56d0140950 0x7f56d00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56d661a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d013ff00 con 0x7f56d00b2e70 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56d661a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d01404c0 con 0x7f56d00cd8c0 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56d661a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d0140dc0 con 0x7f56d0140950 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56d4bb7700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f56d00b2e70 0x7f56d006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51692/0 (socket says 172.21.15.144:51692) 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56d4bb7700 1 -- 172.21.15.144:0/3057927267 learned_addr learned my addr 172.21.15.144:0/3057927267 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:28.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.101+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520402218 0 0) 0x7f56d013ff00 con 0x7f56d00b2e70 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56cf7fe700 1 --2- 172.21.15.144:0/3057927267 >> v2:172.21.15.177:3300/0 conn(0x7f56d0140950 0x7f56d00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b80035b0 con 0x7f56d00b2e70 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1137500879 0 0) 0x7f56b80035b0 con 0x7f56d00b2e70 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 >> v2:172.21.15.177:3300/0 conn(0x7f56d0140950 msgr2=0x7f56d00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 --2- 172.21.15.144:0/3057927267 >> v2:172.21.15.177:3300/0 conn(0x7f56d0140950 0x7f56d00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 >> v1:172.21.15.177:6789/0 conn(0x7f56d00cd8c0 legacy=0x7f56d00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d015aa80 con 0x7f56d00b2e70 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f56c80019f0 con 0x7f56d00b2e70 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 >> v1:172.21.15.144:6789/0 conn(0x7f56d00b2e70 legacy=0x7f56d006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8003a00 0x7f56b8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56b8004850 0x7f56b8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8005200 0x7f56b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0140dc0 con 0x7f56b8005200 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b800c110 con 0x7f56b8003a00 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b800c2d0 con 0x7f56b8004850 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8005200 0x7f56b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56cffff700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8003a00 0x7f56b8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56cf7fe700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56b8004850 0x7f56b8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 -- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56b8004850 msgr2=0x7f56b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56b8004850 0x7f56b8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 -- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8003a00 msgr2=0x7f56b8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8003a00 0x7f56b8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 -- 172.21.15.144:0/3057927267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80035b0 con 0x7f56b8005200 2023-11-28T11:51:28.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 -- 172.21.15.144:0/3057927267 reap_dead start 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8005200 0x7f56b800ba10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56c800a830 tx=0x7f56b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be7a2cc79450912 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.102+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c800a2f0 con 0x7f56b8005200 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c8012600 con 0x7f56b8005200 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56ceffd700 1 -- 172.21.15.144:0/3057927267 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80129b0 con 0x7f56b8005200 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 -- 172.21.15.144:0/3057927267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8005200 msgr2=0x7f56b800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8005200 0x7f56b800ba10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56c800a830 tx=0x7f56b8005880 comp rx=0 tx=0).stop 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 -- 172.21.15.144:0/3057927267 shutdown_connections 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 --2- 172.21.15.144:0/3057927267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56b8005200 0x7f56b800ba10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 -- 172.21.15.144:0/3057927267 >> 172.21.15.144:0/3057927267 conn(0x7f56d006aed0 msgr2=0x7f56d006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 -- 172.21.15.144:0/3057927267 shutdown_connections 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.103+0000 7f56d661a700 1 -- 172.21.15.144:0/3057927267 wait complete. 2023-11-28T11:51:28.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.104+0000 7f56d661a700 1 Processor -- start 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.104+0000 7f56d661a700 1 -- start start 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d661a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8005200 0x7f56d01f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d661a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56d00c6100 0x7f56d00c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d661a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 0x7f56d013d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d661a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d013e360 con 0x7f56d013ccf0 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d661a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00c6ab0 con 0x7f56b8005200 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d661a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d013d6a0 con 0x7f56d00c6100 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d4bb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 0x7f56d013d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d4bb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 0x7f56d013d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34992/0 (socket says 172.21.15.144:34992) 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56d4bb7700 1 -- 172.21.15.144:0/1066080828 learned_addr learned my addr 172.21.15.144:0/1066080828 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56cffff700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8005200 0x7f56d01f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.105+0000 7f56cf7fe700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56d00c6100 0x7f56d00c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d4bb7700 1 -- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56d00c6100 msgr2=0x7f56d00c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56d00c6100 0x7f56d00c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d4bb7700 1 -- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8005200 msgr2=0x7f56d01f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8005200 0x7f56d01f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d4bb7700 1 -- 172.21.15.144:0/1066080828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00b35f0 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d4bb7700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 0x7f56d013d160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56d015e150 tx=0x7f56c8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80017f0 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c8001950 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d00b3310 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c801d660 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.106+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d0140150 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.107+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d01578c0 con 0x7f56d013ccf0 2023-11-28T11:51:28.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.107+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f56c8010070 con 0x7f56d013ccf0 2023-11-28T11:51:28.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.107+0000 7f56ccff9700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f56bc0682f0 0x7f56bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.108+0000 7f56cffff700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f56bc0682f0 0x7f56bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.108+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56c8087c00 con 0x7f56d013ccf0 2023-11-28T11:51:28.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.108+0000 7f56cffff700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f56bc0682f0 0x7f56bc06a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000ef0 tx=0x7f56c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.112+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f56c8054460 con 0x7f56d013ccf0 2023-11-28T11:51:28.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.334+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56d013e4a0 con 0x7f56d013ccf0 2023-11-28T11:51:28.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.335+0000 7f56ccff9700 1 -- 172.21.15.144:0/1066080828 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f56d013e4a0 con 0x7f56d013ccf0 2023-11-28T11:51:28.336 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:28.336 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:51:16.866858+0000","modified":"2023-11-28T11:51:19.878262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f56bc0682f0 msgr2=0x7f56bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f56bc0682f0 0x7f56bc06a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56c4000ef0 tx=0x7f56c4009040 comp rx=0 tx=0).stop 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 msgr2=0x7f56d013d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 0x7f56d013d160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56d015e150 tx=0x7f56c8001d80 comp rx=0 tx=0).stop 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 shutdown_connections 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d013ccf0 0x7f56d013d160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f56bc0682f0 0x7f56bc06a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b8005200 0x7f56d01f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 --2- 172.21.15.144:0/1066080828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56d00c6100 0x7f56d00c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 >> 172.21.15.144:0/1066080828 conn(0x7f56d006aed0 msgr2=0x7f56d00b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 shutdown_connections 2023-11-28T11:51:28.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.337+0000 7f56d661a700 1 -- 172.21.15.144:0/1066080828 wait complete. 2023-11-28T11:51:28.339 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 8 2023-11-28T11:51:28.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:51:16.866858+0000', 'modified': '2023-11-28T11:51:19.878262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6833/144294733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 144294733}, {'type': 'v1', 'addr': '172.21.15.177:6833', 'nonce': 144294733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:51:28.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:51:28.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T11:51:28.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.881+0000 7fade9902700 1 Processor -- start 2023-11-28T11:51:28.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.881+0000 7fade9902700 1 -- start start 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade9902700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fade4071040 0x7fade4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade9902700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade4071950 con 0x7fade4160540 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade9902700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade415d060 con 0x7fade406dee0 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade9902700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade415d340 con 0x7fade4071040 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade37fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fade4160540 0x7fade4160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51712/0 (socket says 172.21.15.144:51712) 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade37fe700 1 -- 172.21.15.144:0/410019346 learned_addr learned my addr 172.21.15.144:0/410019346 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.882+0000 7fade2ffd700 1 --2- 172.21.15.144:0/410019346 >> v2:172.21.15.177:3300/0 conn(0x7fade4071040 0x7fade4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade1ffb700 1 -- 172.21.15.144:0/410019346 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167979264 0 0) 0x7fade4071950 con 0x7fade4160540 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade1ffb700 1 -- 172.21.15.144:0/410019346 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc0035b0 con 0x7fade4160540 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade2ffd700 1 -- 172.21.15.144:0/410019346 >> v1:172.21.15.177:6789/0 conn(0x7fade406dee0 legacy=0x7fade406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade2ffd700 1 -- 172.21.15.144:0/410019346 >> v1:172.21.15.144:6789/0 conn(0x7fade4160540 legacy=0x7fade4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade2ffd700 1 -- 172.21.15.144:0/410019346 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade415d620 con 0x7fade4071040 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade2ffd700 1 --2- 172.21.15.144:0/410019346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4071040 0x7fade4071410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009840 tx=0x7fadd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdc1d47309d71fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade1ffb700 1 -- 172.21.15.144:0/410019346 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8011830 con 0x7fade4071040 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade1ffb700 1 -- 172.21.15.144:0/410019346 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8011990 con 0x7fade4071040 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.883+0000 7fade1ffb700 1 -- 172.21.15.144:0/410019346 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8011d20 con 0x7fade4071040 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 -- 172.21.15.144:0/410019346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4071040 msgr2=0x7fade4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 --2- 172.21.15.144:0/410019346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4071040 0x7fade4071410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009840 tx=0x7fadd8009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 -- 172.21.15.144:0/410019346 shutdown_connections 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 --2- 172.21.15.144:0/410019346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4071040 0x7fade4071410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 -- 172.21.15.144:0/410019346 >> 172.21.15.144:0/410019346 conn(0x7fade40d0000 msgr2=0x7fade40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 -- 172.21.15.144:0/410019346 shutdown_connections 2023-11-28T11:51:28.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.884+0000 7fade9902700 1 -- 172.21.15.144:0/410019346 wait complete. 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.885+0000 7fade9902700 1 Processor -- start 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.885+0000 7fade9902700 1 -- start start 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 0x7fade414c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade4071040 0x7fade40c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4160540 0x7fade4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c5040 con 0x7fade406dee0 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c5320 con 0x7fade4160540 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c5600 con 0x7fade4071040 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 0x7fade414c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 0x7fade414c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35002/0 (socket says 172.21.15.144:35002) 2023-11-28T11:51:28.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 -- 172.21.15.144:0/1136697816 learned_addr learned my addr 172.21.15.144:0/1136697816 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 -- 172.21.15.144:0/1136697816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade4071040 msgr2=0x7fade40c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade4071040 0x7fade40c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 -- 172.21.15.144:0/1136697816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4160540 msgr2=0x7fade4157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4160540 0x7fade4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 -- 172.21.15.144:0/1136697816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80094a0 con 0x7fade406dee0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade2ffd700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 0x7fade414c780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fadd8000c00 tx=0x7fadd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fadcbfff700 1 -- 172.21.15.144:0/1136697816 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80038a0 con 0x7fade406dee0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fadcbfff700 1 -- 172.21.15.144:0/1136697816 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd801e070 con 0x7fade406dee0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fadcbfff700 1 -- 172.21.15.144:0/1136697816 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8003c00 con 0x7fade406dee0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.886+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade415ccd0 con 0x7fade406dee0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.887+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade415d900 con 0x7fade406dee0 2023-11-28T11:51:28.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.888+0000 7fadcbfff700 1 -- 172.21.15.144:0/1136697816 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fadd800f050 con 0x7fade406dee0 2023-11-28T11:51:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.888+0000 7fadcbfff700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fadd00682f0 0x7fadd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.889+0000 7fade27fc700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fadd00682f0 0x7fadd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.889+0000 7fadcbfff700 1 -- 172.21.15.144:0/1136697816 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadd8086c70 con 0x7fade406dee0 2023-11-28T11:51:28.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.889+0000 7fade27fc700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fadd00682f0 0x7fadd006a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fade406efb0 tx=0x7fadd4007a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.890+0000 7fadf5a8bb80 1 Processor -- start 2023-11-28T11:51:28.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.890+0000 7fadf5a8bb80 1 -- start start 2023-11-28T11:51:28.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fe810 0x55e0569d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 0x55e0569febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e056a1a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0568bebc0 con 0x55e0569fc130 2023-11-28T11:51:28.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e05692d350 con 0x55e0569fe810 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e056926880 con 0x55e0569ff120 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fade27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 0x55e0569febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fade27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 0x55e0569febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35012/0 (socket says 172.21.15.144:35012) 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.891+0000 7fade27fc700 1 -- 172.21.15.144:0/316611947 learned_addr learned my addr 172.21.15.144:0/316611947 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade37fe700 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e056a1a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade2ffd700 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fe810 0x55e0569d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade27fc700 1 -- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 msgr2=0x55e056a1a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade27fc700 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e056a1a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade27fc700 1 -- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fe810 msgr2=0x55e0569d4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade27fc700 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fe810 0x55e0569d4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade27fc700 1 -- 172.21.15.144:0/316611947 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0567a85e0 con 0x55e0569fc130 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fade27fc700 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 0x55e0569febe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadd4016710 tx=0x7fadd400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719297b3d802b2c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fadc97fa700 1 -- 172.21.15.144:0/316611947 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd401b070 con 0x55e0569fc130 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fadc97fa700 1 -- 172.21.15.144:0/316611947 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd40162e0 con 0x55e0569fc130 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.892+0000 7fadc97fa700 1 -- 172.21.15.144:0/316611947 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4005330 con 0x55e0569fc130 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/316611947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 msgr2=0x55e0569febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 0x55e0569febe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadd4016710 tx=0x7fadd400f000 comp rx=0 tx=0).stop 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/316611947 shutdown_connections 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e056a1a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fe810 0x55e0569d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/316611947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fc130 0x55e0569febe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/316611947 >> 172.21.15.144:0/316611947 conn(0x55e0569d1d60 msgr2=0x55e0569d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/316611947 shutdown_connections 2023-11-28T11:51:28.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/316611947 wait complete. 2023-11-28T11:51:28.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 -- start start 2023-11-28T11:51:28.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.893+0000 7fadf5a8bb80 1 Processor -- start 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fc130 0x55e0569f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569ff120 0x55e056a292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0567aa8d0 con 0x55e0569fe810 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e056a29a70 con 0x55e0569ff120 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e056a21ba0 con 0x55e0569fc130 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fadf5a8bb80 10 client.0 _ll_register_callbacks cb 0x55e0567cc240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fade27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fade27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35018/0 (socket says 172.21.15.144:35018) 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fade27fc700 1 -- 172.21.15.144:0/1530127497 learned_addr learned my addr 172.21.15.144:0/1530127497 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fade2ffd700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fc130 0x55e0569f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.894+0000 7fade37fe700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569ff120 0x55e056a292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fade27fc700 1 -- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fc130 msgr2=0x55e0569f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fade27fc700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fc130 0x55e0569f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fade27fc700 1 -- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569ff120 msgr2=0x55e056a292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fade27fc700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569ff120 0x55e056a292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fade27fc700 1 -- 172.21.15.144:0/1530127497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade415d060 con 0x55e0569fe810 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fade27fc700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a024e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55e0569d00d0 tx=0x7fadd40010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e922289bdeab525e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadc27fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:51:28.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadc27fc700 1 -- 172.21.15.144:0/1530127497 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4003a90 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadc27fc700 1 -- 172.21.15.144:0/1530127497 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadd4003bf0 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0568f4140 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadc27fc700 1 -- 172.21.15.144:0/1530127497 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd40163a0 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 10 client.4608 fetch_fsmap learned FSMap version 8 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e056754d90 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadc27fc700 1 -- 172.21.15.144:0/1530127497 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0568f4140 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadc27fc700 1 -- 172.21.15.144:0/1530127497 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fadd402a440 con 0x55e0569fe810 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 10 client.4608 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 10 client.4608 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 20 client.4608 populate_metadata read hostname 'smithi144' 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 4 client.4608 mds_command: new command op to 4276 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.895+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 --> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0567aaf10 con 0x55e056a00b70 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.896+0000 7fade2ffd700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.896+0000 7fadc27fc700 10 client.4608 ms_handle_connect on v2:172.21.15.177:6832/144294733 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.896+0000 7fade2ffd700 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a00f40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7520822e7e052d54 server_cookie=e7f80ca27c2e8e8d in_seq=0 out_seq=0 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadc27fc700 1 -- 172.21.15.144:0/1530127497 <== mds.0 v2:172.21.15.177:6832/144294733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55e0567aaf10 con 0x55e056a00b70 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadc27fc700 10 client.4608 handle_command_reply: tid=0 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 client.4608 shutdown 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 msgr2=0x55e056a024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a024e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55e0569d00d0 tx=0x7fadd40010f0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 shutdown_connections 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a00f40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569ff120 0x55e056a292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadc27fc700 0 client.4608 ms_handle_reset on v2:172.21.15.177:6832/144294733 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fc130 0x55e0569f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/1530127497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a024e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 >> 172.21.15.144:0/1530127497 conn(0x55e0569d1d60 msgr2=0x55e0569d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 shutdown_connections 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/1530127497 wait complete. 2023-11-28T11:51:28.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.897+0000 7fadf5a8bb80 20 client.4608 trim_cache size 0 max 16384 2023-11-28T11:51:28.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.914+0000 7fadf5a8bb80 1 Processor -- start 2023-11-28T11:51:28.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.914+0000 7fadf5a8bb80 1 -- start start 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a29060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fe810 0x55e0569f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 0x55e056a2bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0569f65b0 con 0x55e0569ff120 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e056a1f140 con 0x55e0569fc130 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e056a028f0 con 0x55e0569fe810 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 0x55e056a2bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 0x55e056a2bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35028/0 (socket says 172.21.15.144:35028) 2023-11-28T11:51:28.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 -- 172.21.15.144:0/3005516694 learned_addr learned my addr 172.21.15.144:0/3005516694 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 -- 172.21.15.144:0/3005516694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fe810 msgr2=0x55e0569f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fe810 0x55e0569f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 -- 172.21.15.144:0/3005516694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 msgr2=0x55e056a29060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a29060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.915+0000 7fade37fe700 1 -- 172.21.15.144:0/3005516694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0568ed140 con 0x55e0569ff120 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fade37fe700 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 0x55e056a2bdb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faddc001e70 tx=0x7faddc002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94ca8ca8a8bd96b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadc9ffb700 1 -- 172.21.15.144:0/3005516694 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc011ac0 con 0x55e0569ff120 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadc9ffb700 1 -- 172.21.15.144:0/3005516694 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc011c20 con 0x55e0569ff120 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadc9ffb700 1 -- 172.21.15.144:0/3005516694 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc015470 con 0x55e0569ff120 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/3005516694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 msgr2=0x55e056a2bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 0x55e056a2bdb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faddc001e70 tx=0x7faddc002180 comp rx=0 tx=0).stop 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/3005516694 shutdown_connections 2023-11-28T11:51:28.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569ff120 0x55e056a2bdb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a29060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/3005516694 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569fe810 0x55e0569f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/3005516694 >> 172.21.15.144:0/3005516694 conn(0x55e0569d1d60 msgr2=0x55e056a00f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/3005516694 shutdown_connections 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.916+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/3005516694 wait complete. 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.917+0000 7fadf5a8bb80 1 -- start start 2023-11-28T11:51:28.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.917+0000 7fadf5a8bb80 1 Processor -- start 2023-11-28T11:51:28.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a81060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a0f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e0569cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0569d2bc0 con 0x55e0569fe810 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e056a0fd80 con 0x55e0569fc130 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0569d42f0 con 0x55e0569ff120 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 10 client.0 _ll_register_callbacks cb 0x55e0567cc240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a0f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a0f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35032/0 (socket says 172.21.15.144:35032) 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 -- 172.21.15.144:0/988345503 learned_addr learned my addr 172.21.15.144:0/988345503 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade2ffd700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a81060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade37fe700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e0569cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 -- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 msgr2=0x55e0569cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e0569cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 -- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 msgr2=0x55e056a81060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a81060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 -- 172.21.15.144:0/988345503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc00f040 con 0x55e0569fe810 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fade27fc700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a0f290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fadd4001130 tx=0x7fadd4001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0d9a5907eae899 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadc27fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadc27fc700 1 -- 172.21.15.144:0/988345503 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40163a0 con 0x55e0569fe810 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadc27fc700 1 -- 172.21.15.144:0/988345503 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadd402a900 con 0x55e0569fe810 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0568e4160 con 0x55e0569fe810 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadc27fc700 1 -- 172.21.15.144:0/988345503 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd4006860 con 0x55e0569fe810 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 10 client.4614 fetch_fsmap learned FSMap version 8 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.918+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e05677b200 con 0x55e0569fe810 2023-11-28T11:51:28.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadc27fc700 1 -- 172.21.15.144:0/988345503 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e0568e4160 con 0x55e0569fe810 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadc27fc700 1 -- 172.21.15.144:0/988345503 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 8) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fadd40069c0 con 0x55e0569fe810 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 10 client.4614 fetch_fsmap finished waiting for FSMap version 8 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 10 client.4614 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 20 client.4614 populate_metadata read hostname 'smithi144' 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 4 client.4614 mds_command: new command op to 4276 tid=0[{"prefix": "status"}] 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 --> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0568ce170 con 0x55e056a00b70 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fade2ffd700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadc27fc700 10 client.4614 ms_handle_connect on v2:172.21.15.177:6832/144294733 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fade2ffd700 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a018e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698ebbcf8d5f2d00 server_cookie=52281c7a462e116 in_seq=0 out_seq=0 2023-11-28T11:51:28.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadc27fc700 1 -- 172.21.15.144:0/988345503 <== mds.0 v2:172.21.15.177:6832/144294733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55e0568ce170 con 0x55e056a00b70 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadc27fc700 10 client.4614 handle_command_reply: tid=0 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.919+0000 7fadf5a8bb80 1 client.4614 shutdown 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 msgr2=0x55e056a0f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a0f290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fadd4001130 tx=0x7fadd4001080 comp rx=0 tx=0).stop 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 shutdown_connections 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:6832/144294733,v1:172.21.15.177:6833/144294733] conn(0x55e056a00b70 0x55e056a018e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55e0569ff120 0x55e0569cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadc27fc700 0 client.4614 ms_handle_reset on v2:172.21.15.177:6832/144294733 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e0569fc130 0x55e056a81060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 --2- 172.21.15.144:0/988345503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0569fe810 0x55e056a0f290 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 >> 172.21.15.144:0/988345503 conn(0x55e0569d1d60 msgr2=0x55e0569d5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 shutdown_connections 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 1 -- 172.21.15.144:0/988345503 wait complete. 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fadf5a8bb80 20 client.4614 trim_cache size 0 max 16384 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout: "cluster_fsid": "af726c13-c3f1-4279-9fca-12c6e4621b18", 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout: "whoami": 0, 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4276, 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout: "want_state": "up:active", 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout: "state": "up:active", 2023-11-28T11:51:28.929 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs", 2023-11-28T11:51:28.930 INFO:teuthology.orchestra.run.smithi144.stdout: "rank_uptime": 12.051355625999999, 2023-11-28T11:51:28.930 INFO:teuthology.orchestra.run.smithi144.stdout: "mdsmap_epoch": 8, 2023-11-28T11:51:28.930 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch": 22, 2023-11-28T11:51:28.930 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch_barrier": 22, 2023-11-28T11:51:28.930 INFO:teuthology.orchestra.run.smithi144.stdout: "uptime": 17.351418852999998 2023-11-28T11:51:28.930 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fadd00682f0 msgr2=0x7fadd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fadd00682f0 0x7fadd006a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fade406efb0 tx=0x7fadd4007a00 comp rx=0 tx=0).stop 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 msgr2=0x7fade414c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 0x7fade414c780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fadd8000c00 tx=0x7fadd8004750 comp rx=0 tx=0).stop 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 shutdown_connections 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade406dee0 0x7fade414c780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fadd00682f0 0x7fadd006a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade4160540 0x7fade4157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 --2- 172.21.15.144:0/1136697816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fade4071040 0x7fade40c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:28.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 >> 172.21.15.144:0/1136697816 conn(0x7fade40d0000 msgr2=0x7fade406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:28.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 shutdown_connections 2023-11-28T11:51:28.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:28.920+0000 7fade9902700 1 -- 172.21.15.144:0/1136697816 wait complete. 2023-11-28T11:51:28.933 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T11:51:28.956 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T11:51:28.956 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T11:51:28.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:28.957 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:51:28.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:28.976 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-11-28T11:51:29.043 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-11-28T11:51:29.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:51:29.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.045 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:51:29.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.104 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-11-28T11:51:29.171 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-11-28T11:51:29.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:51:29.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.173 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T11:51:29.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.193 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T11:51:29.269 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-11-28T11:51:29.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:51:29.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.271 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T11:51:29.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.331 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T11:51:29.401 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-11-28T11:51:29.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:51:29.402 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T11:51:29.402 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:51:29.403 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:51:29.403 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:51:29.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.523+0000 7f6a390d4700 1 Processor -- start 2023-11-28T11:51:29.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.523+0000 7f6a390d4700 1 -- start start 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a390d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a34158ce0 0x7f6a3415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a390d4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3406bf10 con 0x7f6a340ce4b0 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a390d4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3415d730 con 0x7f6a3406fb30 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a390d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341590b0 con 0x7f6a34158ce0 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a3359e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6a340ce4b0 0x7f6a3406c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51754/0 (socket says 172.21.15.144:51754) 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a3359e700 1 -- 172.21.15.144:0/2202244126 learned_addr learned my addr 172.21.15.144:0/2202244126 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a32d9d700 1 --2- 172.21.15.144:0/2202244126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a34158ce0 0x7f6a3415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987654520 0 0) 0x7f6a3406bf10 con 0x7f6a340ce4b0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a200035b0 con 0x7f6a340ce4b0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709914133 0 0) 0x7f6a3415d730 con 0x7f6a3406fb30 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3406bf10 con 0x7f6a3406fb30 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.524+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3830839809 0 0) 0x7f6a200035b0 con 0x7f6a340ce4b0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a32d9d700 1 -- 172.21.15.144:0/2202244126 >> v1:172.21.15.177:6789/0 conn(0x7f6a3406fb30 legacy=0x7f6a340cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a32d9d700 1 -- 172.21.15.144:0/2202244126 >> v1:172.21.15.144:6789/0 conn(0x7f6a340ce4b0 legacy=0x7f6a3406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a32d9d700 1 -- 172.21.15.144:0/2202244126 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340cb0d0 con 0x7f6a34158ce0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a32d9d700 1 --2- 172.21.15.144:0/2202244126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a34158ce0 0x7f6a3415d1f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00aed0 tx=0x7f6a2c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8783e4ae2a277ad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00f070 con 0x7f6a34158ce0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00a2e0 con 0x7f6a34158ce0 2023-11-28T11:51:29.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a31d9b700 1 -- 172.21.15.144:0/2202244126 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24001670 con 0x7f6a34158ce0 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 -- 172.21.15.144:0/2202244126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a34158ce0 msgr2=0x7f6a3415d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 --2- 172.21.15.144:0/2202244126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a34158ce0 0x7f6a3415d1f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00aed0 tx=0x7f6a2c00c330 comp rx=0 tx=0).stop 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 -- 172.21.15.144:0/2202244126 shutdown_connections 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 --2- 172.21.15.144:0/2202244126 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a34158ce0 0x7f6a3415d1f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 -- 172.21.15.144:0/2202244126 >> 172.21.15.144:0/2202244126 conn(0x7f6a340cb670 msgr2=0x7f6a340cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 -- 172.21.15.144:0/2202244126 shutdown_connections 2023-11-28T11:51:29.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.525+0000 7f6a390d4700 1 -- 172.21.15.144:0/2202244126 wait complete. 2023-11-28T11:51:29.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.528+0000 7f6a390d4700 1 Processor -- start 2023-11-28T11:51:29.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.528+0000 7f6a390d4700 1 -- start start 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.528+0000 7f6a390d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 0x7f6a340c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ce4b0 0x7f6a340b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a34158ce0 0x7f6a34153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c8cb0 con 0x7f6a3406fb30 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340b41e0 con 0x7f6a34158ce0 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a34153db0 con 0x7f6a340ce4b0 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 0x7f6a340c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 0x7f6a340c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35044/0 (socket says 172.21.15.144:35044) 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 -- 172.21.15.144:0/4009649846 learned_addr learned my addr 172.21.15.144:0/4009649846 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:29.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a3359e700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a34158ce0 0x7f6a34153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a3259c700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ce4b0 0x7f6a340b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 -- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ce4b0 msgr2=0x7f6a340b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ce4b0 0x7f6a340b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 -- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a34158ce0 msgr2=0x7f6a34153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a34158ce0 0x7f6a34153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 -- 172.21.15.144:0/4009649846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c00ab80 con 0x7f6a3406fb30 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a3259c700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ce4b0 0x7f6a340b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a3359e700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a34158ce0 0x7f6a34153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a32d9d700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 0x7f6a340c8770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00aea0 tx=0x7f6a2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c004240 con 0x7f6a3406fb30 2023-11-28T11:51:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0043a0 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3406b8c0 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c004750 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.529+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340cb3b0 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.530+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3406bba0 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.531+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a2c014440 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.531+0000 7f6a2b7fe700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a140682f0 0x7f6a1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.531+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a2c086f10 con 0x7f6a3406fb30 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.531+0000 7f6a3259c700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a140682f0 0x7f6a1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:29.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.531+0000 7f6a3259c700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a140682f0 0x7f6a1406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a340c5910 tx=0x7f6a1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.535+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a2c053710 con 0x7f6a3406fb30 2023-11-28T11:51:29.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.762+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6a340b4b70 con 0x7f6a3406fb30 2023-11-28T11:51:29.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.762+0000 7f6a2b7fe700 1 -- 172.21.15.144:0/4009649846 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a2c020020 con 0x7f6a3406fb30 2023-11-28T11:51:29.768 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:51:29.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.764+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a140682f0 msgr2=0x7f6a1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.764+0000 7f6a390d4700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a140682f0 0x7f6a1406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a340c5910 tx=0x7f6a1c009040 comp rx=0 tx=0).stop 2023-11-28T11:51:29.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.764+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 msgr2=0x7f6a340c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:29.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.764+0000 7f6a390d4700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 0x7f6a340c8770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00aea0 tx=0x7f6a2c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 shutdown_connections 2023-11-28T11:51:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a3406fb30 0x7f6a340c8770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a140682f0 0x7f6a1406a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a34158ce0 0x7f6a34153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 --2- 172.21.15.144:0/4009649846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a340ce4b0 0x7f6a340b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 >> 172.21.15.144:0/4009649846 conn(0x7f6a340cb670 msgr2=0x7f6a34157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:29.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 shutdown_connections 2023-11-28T11:51:29.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:29.765+0000 7f6a390d4700 1 -- 172.21.15.144:0/4009649846 wait complete. 2023-11-28T11:51:29.778 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:51:29.779 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:51:29.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.780 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-11-28T11:51:29.797 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:29.798 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:51:29.799 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:29.799 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:51:29.799 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42413sec preferred_lft 42413sec 2023-11-28T11:51:29.799 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1121:9a48:f2a:7a1a/64 scope link noprefixroute 2023-11-28T11:51:29.799 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:29.800 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:51:29.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:51:29.800 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:29.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:29.976 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:51:30.003 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T11:51:30.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.005 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-11-28T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-11-28T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:51:30.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.084 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:30.084 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:30.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:51:30.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:51:30.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:51:30.084 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:30.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.328 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:51:30.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.346 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2023-11-28T11:51:30.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.402 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:30.402 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:30.402 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:51:30.403 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T11:51:30.403 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:30.499 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T11:51:30.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.499 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:30.499 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:30.499 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T11:51:30.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:51:30.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T11:51:30.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T11:51:30.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T11:51:30.500 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:30.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.701 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:30.701 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:30.701 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-11-28T11:51:30.701 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:51:30.701 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:30.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:30.803 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:51:30.823 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:51:30.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:30.823 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:51:30.879 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:51:30.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:30.880 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:51:30.973 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:51:30.973 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:51:30.973 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:51:31.086 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:51:31.086 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:51:31.086 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:51:31.086 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:31.086 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:51:31.157 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:51:31.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.157 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:51:31.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.184 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:31.273 INFO:teuthology.orchestra.run.smithi144.stdout:client4469 2023-11-28T11:51:31.273 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:31.273 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:31.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.290 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:31.380 INFO:teuthology.orchestra.run.smithi144.stdout:client4469 2023-11-28T11:51:31.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.381 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:31.426 INFO:teuthology.orchestra.run.smithi144.stdout:client4469 2023-11-28T11:51:31.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.427 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/af726c13-c3f1-4279-9fca-12c6e4621b18.client4469/status') 2023-11-28T11:51:31.479 INFO:teuthology.orchestra.run.smithi144.stdout:instance: client.4469 (3)172.21.15.144:0/2766862284 2023-11-28T11:51:31.479 INFO:teuthology.orchestra.run.smithi144.stdout:blocklisted: false 2023-11-28T11:51:31.479 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records in 2023-11-28T11:51:31.479 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records out 2023-11-28T11:51:31.479 INFO:teuthology.orchestra.run.smithi144.stderr:71 bytes copied, 2.3065e-05 s, 3.1 MB/s 2023-11-28T11:51:31.480 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-28T11:51:31.480 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:51:31.480 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:51:31.481 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:51:31.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.601+0000 7fa80c126700 1 Processor -- start 2023-11-28T11:51:31.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.602+0000 7fa80c126700 1 -- start start 2023-11-28T11:51:31.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.602+0000 7fa80c126700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa804159860 0x7fa8040c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.602+0000 7fa80c126700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa8040b0d80 0x7fa8040c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.602+0000 7fa80c126700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8040ccc00 con 0x7fa804159490 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.602+0000 7fa80c126700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8040ccec0 con 0x7fa8040b0d80 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80c126700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8040cd1a0 con 0x7fa804159860 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa8040b0d80 0x7fa8040c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa8040b0d80 0x7fa8040c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35056/0 (socket says 172.21.15.144:35056) 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 -- 172.21.15.144:0/2135008130 learned_addr learned my addr 172.21.15.144:0/2135008130 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa8096c1700 1 --2- 172.21.15.144:0/2135008130 >> v2:172.21.15.177:3300/0 conn(0x7fa804159860 0x7fa8040c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 -- 172.21.15.144:0/2135008130 >> v2:172.21.15.177:3300/0 conn(0x7fa804159860 msgr2=0x7fa8040c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 --2- 172.21.15.144:0/2135008130 >> v2:172.21.15.177:3300/0 conn(0x7fa804159860 0x7fa8040c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 -- 172.21.15.144:0/2135008130 >> v1:172.21.15.177:6789/0 conn(0x7fa804159490 legacy=0x7fa8040b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:31.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 -- 172.21.15.144:0/2135008130 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040cd480 con 0x7fa8040b0d80 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa8096c1700 1 --2- 172.21.15.144:0/2135008130 >> v2:172.21.15.177:3300/0 conn(0x7fa804159860 0x7fa8040c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa80a6c3700 1 --2- 172.21.15.144:0/2135008130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8040b0d80 0x7fa8040c9a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa8040cb5d0 tx=0x7fa7fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4742b6643c9f3499 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa808ec0700 1 -- 172.21.15.144:0/2135008130 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00e070 con 0x7fa8040b0d80 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa808ec0700 1 -- 172.21.15.144:0/2135008130 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc00b9c0 con 0x7fa8040b0d80 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.603+0000 7fa808ec0700 1 -- 172.21.15.144:0/2135008130 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00bd70 con 0x7fa8040b0d80 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 -- 172.21.15.144:0/2135008130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8040b0d80 msgr2=0x7fa8040c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 --2- 172.21.15.144:0/2135008130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8040b0d80 0x7fa8040c9a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa8040cb5d0 tx=0x7fa7fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 -- 172.21.15.144:0/2135008130 shutdown_connections 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 --2- 172.21.15.144:0/2135008130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8040b0d80 0x7fa8040c9a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 --2- 172.21.15.144:0/2135008130 >> v2:172.21.15.177:3300/0 conn(0x7fa804159860 0x7fa8040c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 -- 172.21.15.144:0/2135008130 >> 172.21.15.144:0/2135008130 conn(0x7fa8040cb670 msgr2=0x7fa8040cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 -- 172.21.15.144:0/2135008130 shutdown_connections 2023-11-28T11:51:31.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.604+0000 7fa80c126700 1 -- 172.21.15.144:0/2135008130 wait complete. 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.605+0000 7fa80c126700 1 Processor -- start 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.605+0000 7fa80c126700 1 -- start start 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80c126700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8040b0d80 0x7fa8041f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80c126700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 0x7fa8040c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80c126700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8040c8b40 0x7fa8040b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80c126700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8041f2570 con 0x7fa804159490 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80c126700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040c6170 con 0x7fa8040c8b40 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80c126700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040c9020 con 0x7fa8040b0d80 2023-11-28T11:51:31.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 0x7fa8040c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 0x7fa8040c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35062/0 (socket says 172.21.15.144:35062) 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 -- 172.21.15.144:0/814306210 learned_addr learned my addr 172.21.15.144:0/814306210 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80a6c3700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8040c8b40 0x7fa8040b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa809ec2700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8040b0d80 0x7fa8041f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 -- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8040b0d80 msgr2=0x7fa8041f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8040b0d80 0x7fa8041f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 -- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8040c8b40 msgr2=0x7fa8040b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8040c8b40 0x7fa8040b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 -- 172.21.15.144:0/814306210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc009b30 con 0x7fa804159490 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa80a6c3700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8040c8b40 0x7fa8040b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.606+0000 7fa8096c1700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 0x7fa8040c5c30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e760 tx=0x7fa80000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:31.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.607+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800013610 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.607+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa800013770 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.607+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8040c9f40 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.607+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800013b20 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.607+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8040cd760 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.607+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8040cdc80 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.608+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa800023070 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.608+0000 7fa7fa7fc700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7e80682f0 0x7fa7e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.608+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa800085880 con 0x7fa804159490 2023-11-28T11:51:31.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.608+0000 7fa809ec2700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7e80682f0 0x7fa7e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:31.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.609+0000 7fa809ec2700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7e80682f0 0x7fa7e806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4000f60 tx=0x7fa7f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:31.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.613+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8000520e0 con 0x7fa804159490 2023-11-28T11:51:31.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.840+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa804156a60 con 0x7fa804159490 2023-11-28T11:51:31.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.840+0000 7fa7fa7fc700 1 -- 172.21.15.144:0/814306210 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa800055700 con 0x7fa804159490 2023-11-28T11:51:31.841 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:51:31.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7e80682f0 msgr2=0x7fa7e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:31.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7e80682f0 0x7fa7e806a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4000f60 tx=0x7fa7f4009040 comp rx=0 tx=0).stop 2023-11-28T11:51:31.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 msgr2=0x7fa8040c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:31.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 0x7fa8040c5c30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e760 tx=0x7fa80000eb20 comp rx=0 tx=0).stop 2023-11-28T11:51:31.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 shutdown_connections 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa7e80682f0 0x7fa7e806a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8040c8b40 0x7fa8040b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8040b0d80 0x7fa8041f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 --2- 172.21.15.144:0/814306210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa804159490 0x7fa8040c5c30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 >> 172.21.15.144:0/814306210 conn(0x7fa8040cb670 msgr2=0x7fa804154430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 shutdown_connections 2023-11-28T11:51:31.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:31.843+0000 7fa80c126700 1 -- 172.21.15.144:0/814306210 wait complete. 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:51:31.856 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:51:31.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.857 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-11-28T11:51:31.874 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:31.874 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:51:31.875 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42411sec preferred_lft 42411sec 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1121:9a48:f2a:7a1a/64 scope link noprefixroute 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether ee:83:c8:ce:35:5d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::fccf:aaff:fe54:493c/64 scope link 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether ee:83:c8:ce:35:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec83:c8ff:fece:355d/64 scope link tentative 2023-11-28T11:51:31.876 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:31.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.877 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:51:31.939 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T11:51:31.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:31.941 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-11-28T11:51:32.000 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-11-28T11:51:32.001 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:51:32.001 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:51:32.001 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:51:32.001 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:51:32.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.002 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:32.002 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:32.002 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:51:32.002 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:51:32.002 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:51:32.002 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:32.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.105 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:51:32.122 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:51:32.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.123 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2023-11-28T11:51:32.179 INFO:teuthology.orchestra.run.smithi144.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T11:51:32.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.180 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:32.180 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:32.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:51:32.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T11:51:32.180 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:32.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.273 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:51:32.310 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:32.310 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 02:22:d6:23:86:83 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::22:d6ff:fe23:8683/64 scope link tentative 2023-11-28T11:51:32.311 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.328 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:51:32.403 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:32.403 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:32.403 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 02:22:d6:23:86:83 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::22:d6ff:fe23:8683/64 scope link tentative 2023-11-28T11:51:32.404 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:32.420 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T11:51:32.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T11:51:32.421 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:32.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.679 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:51:32.679 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:51:32.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.1 up 2023-11-28T11:51:32.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:51:32.679 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:51:32.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:32.789 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T11:51:32.808 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:51:32.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:32.809 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:51:32.866 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:51:32.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:32.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:51:32.952 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:51:32.952 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:51:32.952 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:51:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:51:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:51:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:51:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:32.961 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:33.036 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:51:33.037 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:51:33.039 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:51:33.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.039 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:51:33.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.065 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:33.138 INFO:teuthology.orchestra.run.smithi144.stdout:client4472 2023-11-28T11:51:33.138 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:33.139 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:33.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.198 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:33.270 INFO:teuthology.orchestra.run.smithi144.stdout:client4472 2023-11-28T11:51:33.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.271 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:33.342 INFO:teuthology.orchestra.run.smithi144.stdout:client4472 2023-11-28T11:51:33.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.343 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/af726c13-c3f1-4279-9fca-12c6e4621b18.client4472/status') 2023-11-28T11:51:33.423 INFO:teuthology.orchestra.run.smithi144.stdout:instance: client.4472 (3)172.21.15.144:0/880295854 2023-11-28T11:51:33.423 INFO:teuthology.orchestra.run.smithi144.stdout:blocklisted: false 2023-11-28T11:51:33.423 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records in 2023-11-28T11:51:33.423 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records out 2023-11-28T11:51:33.423 INFO:teuthology.orchestra.run.smithi144.stderr:70 bytes copied, 3.8389e-05 s, 1.8 MB/s 2023-11-28T11:51:33.424 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-28T11:51:33.424 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:51:33.424 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:51:33.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:51:33.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.584+0000 7fa68286a700 1 Processor -- start 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.584+0000 7fa68286a700 1 -- start start 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa68286a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa67c06ec90 0x7fa67c065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa68286a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa67c0cec70 0x7fa67c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa68286a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c06e240 con 0x7fa67c0c9f10 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa68286a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c066580 con 0x7fa67c0cec70 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa68286a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c0cded0 con 0x7fa67c06ec90 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa67c0cec70 0x7fa67c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa67c0cec70 0x7fa67c0ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35074/0 (socket says 172.21.15.144:35074) 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 -- 172.21.15.144:0/591491773 learned_addr learned my addr 172.21.15.144:0/591491773 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67bfff700 1 --2- 172.21.15.144:0/591491773 >> v2:172.21.15.177:3300/0 conn(0x7fa67c06ec90 0x7fa67c065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:33.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 -- 172.21.15.144:0/591491773 >> v2:172.21.15.177:3300/0 conn(0x7fa67c06ec90 msgr2=0x7fa67c065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 --2- 172.21.15.144:0/591491773 >> v2:172.21.15.177:3300/0 conn(0x7fa67c06ec90 0x7fa67c065ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 -- 172.21.15.144:0/591491773 >> v1:172.21.15.177:6789/0 conn(0x7fa67c0c9f10 legacy=0x7fa67c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 -- 172.21.15.144:0/591491773 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0ce1b0 con 0x7fa67c0cec70 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67bfff700 1 --2- 172.21.15.144:0/591491773 >> v2:172.21.15.177:3300/0 conn(0x7fa67c06ec90 0x7fa67c065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.585+0000 7fa67b7fe700 1 --2- 172.21.15.144:0/591491773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0ca9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa67c073d40 tx=0x7fa670009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952d1ceab4d9017e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa67affd700 1 -- 172.21.15.144:0/591491773 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670004230 con 0x7fa67c0cec70 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa67affd700 1 -- 172.21.15.144:0/591491773 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670004390 con 0x7fa67c0cec70 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa67affd700 1 -- 172.21.15.144:0/591491773 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700036a0 con 0x7fa67c0cec70 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 -- 172.21.15.144:0/591491773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 msgr2=0x7fa67c0ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 --2- 172.21.15.144:0/591491773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0ca9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa67c073d40 tx=0x7fa670009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 -- 172.21.15.144:0/591491773 shutdown_connections 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 --2- 172.21.15.144:0/591491773 >> v2:172.21.15.177:3300/0 conn(0x7fa67c06ec90 0x7fa67c065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 --2- 172.21.15.144:0/591491773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0ca9d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 -- 172.21.15.144:0/591491773 >> 172.21.15.144:0/591491773 conn(0x7fa67c0b12b0 msgr2=0x7fa67c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 -- 172.21.15.144:0/591491773 shutdown_connections 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.586+0000 7fa68286a700 1 -- 172.21.15.144:0/591491773 wait complete. 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.588+0000 7fa68286a700 1 Processor -- start 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.588+0000 7fa68286a700 1 -- start start 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa68286a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa67c06ec90 0x7fa67c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa68286a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa67c0c9f10 0x7fa67c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa68286a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:33.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa68286a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c15a290 con 0x7fa67c0cec70 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa68286a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c15a530 con 0x7fa67c0c9f10 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa68286a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c15a810 con 0x7fa67c06ec90 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35082/0 (socket says 172.21.15.144:35082) 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 -- 172.21.15.144:0/4107982802 learned_addr learned my addr 172.21.15.144:0/4107982802 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa67b7fe700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa67c0c9f10 0x7fa67c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa67bfff700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa67c06ec90 0x7fa67c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 -- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa67c06ec90 msgr2=0x7fa67c153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa67c06ec90 0x7fa67c153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 -- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa67c0c9f10 msgr2=0x7fa67c157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa67c0c9f10 0x7fa67c157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 -- 172.21.15.144:0/4107982802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670009550 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa680e07700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0b0570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa67400eeb0 tx=0x7fa67400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.589+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674011670 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.590+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6740117d0 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.590+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c0ce770 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.590+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674011b80 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.590+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c0ce490 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.590+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c0cb6c0 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.591+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa674011ce0 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.591+0000 7fa678ff9700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6680682f0 0x7fa66806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.591+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa674088ab0 con 0x7fa67c0cec70 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.591+0000 7fa67bfff700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6680682f0 0x7fa66806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:33.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.592+0000 7fa67bfff700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6680682f0 0x7fa66806a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa66c006fd0 tx=0x7fa66c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:33.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.596+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa674055330 con 0x7fa67c0cec70 2023-11-28T11:51:33.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.822+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa67c0b2840 con 0x7fa67c0cec70 2023-11-28T11:51:33.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.822+0000 7fa678ff9700 1 -- 172.21.15.144:0/4107982802 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa67400ce60 con 0x7fa67c0cec70 2023-11-28T11:51:33.823 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6680682f0 msgr2=0x7fa66806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6680682f0 0x7fa66806a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa66c006fd0 tx=0x7fa66c008040 comp rx=0 tx=0).stop 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 msgr2=0x7fa67c0b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0b0570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa67400eeb0 tx=0x7fa67400ef90 comp rx=0 tx=0).stop 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 shutdown_connections 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa67c0cec70 0x7fa67c0b0570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6680682f0 0x7fa66806a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa67c06ec90 0x7fa67c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 --2- 172.21.15.144:0/4107982802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa67c0c9f10 0x7fa67c157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:33.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 >> 172.21.15.144:0/4107982802 conn(0x7fa67c0b12b0 msgr2=0x7fa67c0af380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:33.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 shutdown_connections 2023-11-28T11:51:33.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:33.824+0000 7fa68286a700 1 -- 172.21.15.144:0/4107982802 wait complete. 2023-11-28T11:51:33.837 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:51:33.837 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:51:33.837 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-11-28T11:51:33.837 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T11:51:33.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:51:33.837 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:51:33.838 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:51:33.838 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:51:33.838 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:51:33.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.838 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:33.859 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42410sec preferred_lft 42410sec 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::cb7:de77:3153:2cb9/64 scope link noprefixroute 2023-11-28T11:51:33.860 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:33.861 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:51:33.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr flush dev ceph-brx 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set ceph-brx up 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:51:33.861 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:34.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.041 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:51:34.074 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-11-28T11:51:34.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.075 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-11-28T11:51:34.152 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-11-28T11:51:34.153 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:51:34.153 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:51:34.153 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:51:34.153 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:51:34.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.153 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:34.153 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:34.153 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:51:34.153 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:51:34.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:51:34.154 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:34.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.414 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T11:51:34.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.431 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-11-28T11:51:34.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.487 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:34.487 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:34.487 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:51:34.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T11:51:34.488 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:34.581 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T11:51:34.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T11:51:34.582 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:34.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.781 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:34.781 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:34.781 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2023-11-28T11:51:34.781 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:51:34.781 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:34.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:34.883 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T11:51:34.903 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:51:34.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:34.904 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:51:34.963 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:51:34.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:34.964 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:51:35.060 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:51:35.061 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:51:35.061 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T11:51:35.158 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:51:35.159 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:51:35.159 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T11:51:35.159 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:35.159 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:35.206 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:51:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:51:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T11:51:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:35.208 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:51:35.208 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:51:35.208 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:51:35.221 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:51:35.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:35.222 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:51:35.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:35.249 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:35.321 INFO:teuthology.orchestra.run.smithi177.stdout:client4475 2023-11-28T11:51:35.322 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:35.322 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:35.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:35.380 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:35.453 INFO:teuthology.orchestra.run.smithi177.stdout:client4475 2023-11-28T11:51:35.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:35.454 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:35.526 INFO:teuthology.orchestra.run.smithi177.stdout:client4475 2023-11-28T11:51:35.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:35.526 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/af726c13-c3f1-4279-9fca-12c6e4621b18.client4475/status') 2023-11-28T11:51:35.609 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4475 (3)192.168.0.1:0/2539410918 2023-11-28T11:51:35.610 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2023-11-28T11:51:35.611 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2023-11-28T11:51:35.611 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2023-11-28T11:51:35.611 INFO:teuthology.orchestra.run.smithi177.stderr:69 bytes copied, 2.0312e-05 s, 3.4 MB/s 2023-11-28T11:51:35.612 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-28T11:51:35.612 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:51:35.612 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:51:35.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:51:35.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.747+0000 7faaebe95700 1 Processor -- start 2023-11-28T11:51:35.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.747+0000 7faaebe95700 1 -- start start 2023-11-28T11:51:35.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faaebe95700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faae406dee0 0x7faae406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:35.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faaebe95700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7faae4160540 0x7faae4160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faaebe95700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae415d060 con 0x7faae4071040 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faaebe95700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae415d340 con 0x7faae406dee0 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faaebe95700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae415d620 con 0x7faae4160540 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faae406dee0 0x7faae406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faae406dee0 0x7faae406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35088/0 (socket says 172.21.15.144:35088) 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 -- 172.21.15.144:0/3774077823 learned_addr learned my addr 172.21.15.144:0/3774077823 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faaea432700 1 --2- 172.21.15.144:0/3774077823 >> v2:172.21.15.177:3300/0 conn(0x7faae4160540 0x7faae4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 -- 172.21.15.144:0/3774077823 >> v2:172.21.15.177:3300/0 conn(0x7faae4160540 msgr2=0x7faae4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:35.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 --2- 172.21.15.144:0/3774077823 >> v2:172.21.15.177:3300/0 conn(0x7faae4160540 0x7faae4160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 -- 172.21.15.144:0/3774077823 >> v1:172.21.15.177:6789/0 conn(0x7faae4071040 legacy=0x7faae4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 -- 172.21.15.144:0/3774077823 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae415d900 con 0x7faae406dee0 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.748+0000 7faae9430700 1 --2- 172.21.15.144:0/3774077823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae406dee0 0x7faae406e330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faae40c3a10 tx=0x7faad4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11fa95fe6d5d8a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faae8c2f700 1 -- 172.21.15.144:0/3774077823 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4004230 con 0x7faae406dee0 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faae8c2f700 1 -- 172.21.15.144:0/3774077823 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad4004390 con 0x7faae406dee0 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faae8c2f700 1 -- 172.21.15.144:0/3774077823 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad40036a0 con 0x7faae406dee0 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 -- 172.21.15.144:0/3774077823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae406dee0 msgr2=0x7faae406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 --2- 172.21.15.144:0/3774077823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae406dee0 0x7faae406e330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faae40c3a10 tx=0x7faad4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 -- 172.21.15.144:0/3774077823 shutdown_connections 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 --2- 172.21.15.144:0/3774077823 >> v2:172.21.15.177:3300/0 conn(0x7faae4160540 0x7faae4160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 --2- 172.21.15.144:0/3774077823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae406dee0 0x7faae406e330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 -- 172.21.15.144:0/3774077823 >> 172.21.15.144:0/3774077823 conn(0x7faae40d0000 msgr2=0x7faae40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 -- 172.21.15.144:0/3774077823 shutdown_connections 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.749+0000 7faaebe95700 1 -- 172.21.15.144:0/3774077823 wait complete. 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.750+0000 7faaebe95700 1 Processor -- start 2023-11-28T11:51:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.750+0000 7faaebe95700 1 -- start start 2023-11-28T11:51:35.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaebe95700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae406dee0 0x7faae4159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:35.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaebe95700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae4071040 0x7faae4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:35.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaebe95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 0x7faae40cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:35.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaebe95700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4156db0 con 0x7faae4160540 2023-11-28T11:51:35.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaebe95700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b2e30 con 0x7faae4071040 2023-11-28T11:51:35.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaebe95700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b3110 con 0x7faae406dee0 2023-11-28T11:51:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaea432700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 0x7faae40cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faae9430700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae4071040 0x7faae4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaea432700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 0x7faae40cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35096/0 (socket says 172.21.15.144:35096) 2023-11-28T11:51:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faaea432700 1 -- 172.21.15.144:0/2941581701 learned_addr learned my addr 172.21.15.144:0/2941581701 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.751+0000 7faae9c31700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae406dee0 0x7faae4159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaea432700 1 -- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae406dee0 msgr2=0x7faae4159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaea432700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae406dee0 0x7faae4159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaea432700 1 -- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae4071040 msgr2=0x7faae4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaea432700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae4071040 0x7faae4159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaea432700 1 -- 172.21.15.144:0/2941581701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad4009550 con 0x7faae4160540 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaea432700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 0x7faae40cf060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faadc00cc30 tx=0x7faadc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00aa50 con 0x7faae4160540 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc00abb0 con 0x7faae4160540 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc003a20 con 0x7faae4160540 2023-11-28T11:51:35.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae415ccd0 con 0x7faae4160540 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.752+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae406e870 con 0x7faae4160540 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.753+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae406a270 con 0x7faae4160540 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.753+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faadc01f070 con 0x7faae4160540 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.754+0000 7faada7fc700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faac80682f0 0x7faac806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.754+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faadc085bf0 con 0x7faae4160540 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.754+0000 7faae9c31700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faac80682f0 0x7faac806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.754+0000 7faae9c31700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faac80682f0 0x7faac806a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faae0009960 tx=0x7faae00090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.758+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faadc0523f0 con 0x7faae4160540 2023-11-28T11:51:35.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.982+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faae4158a20 con 0x7faae4160540 2023-11-28T11:51:35.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.982+0000 7faada7fc700 1 -- 172.21.15.144:0/2941581701 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7faadc01d070 con 0x7faae4160540 2023-11-28T11:51:35.984 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faac80682f0 msgr2=0x7faac806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faac80682f0 0x7faac806a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faae0009960 tx=0x7faae00090e0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 msgr2=0x7faae40cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 0x7faae40cf060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faadc00cc30 tx=0x7faadc00cf40 comp rx=0 tx=0).stop 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 shutdown_connections 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faae4160540 0x7faae40cf060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7faac80682f0 0x7faac806a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faae406dee0 0x7faae4159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 --2- 172.21.15.144:0/2941581701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faae4071040 0x7faae4159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:35.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 >> 172.21.15.144:0/2941581701 conn(0x7faae40d0000 msgr2=0x7faae415e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:35.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 shutdown_connections 2023-11-28T11:51:35.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:35.984+0000 7faaebe95700 1 -- 172.21.15.144:0/2941581701 wait complete. 2023-11-28T11:51:35.998 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:51:35.998 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:51:35.998 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-11-28T11:51:35.998 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T11:51:35.998 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:51:35.999 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:51:35.999 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:51:35.999 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:51:35.999 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:51:35.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:35.999 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-11-28T11:51:36.019 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:36.019 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:36.020 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:36.020 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.020 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:36.020 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.020 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:36.020 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:36.021 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42408sec preferred_lft 42408sec 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::cb7:de77:3153:2cb9/64 scope link noprefixroute 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether fa:73:38:c1:d5:e9 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::488e:eeff:fe2d:1c6b/64 scope link 2023-11-28T11:51:36.022 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.023 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:51:36.023 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether fa:73:38:c1:d5:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:51:36.023 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::f873:38ff:fec1:d5e9/64 scope link 2023-11-28T11:51:36.023 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.023 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:51:36.090 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-11-28T11:51:36.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.091 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-11-28T11:51:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-11-28T11:51:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:51:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:51:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:51:36.153 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:51:36.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.154 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:36.154 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:36.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:51:36.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:51:36.154 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:51:36.154 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:36.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.262 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T11:51:36.282 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:51:36.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.282 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-11-28T11:51:36.340 INFO:teuthology.orchestra.run.smithi177.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T11:51:36.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.341 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:36.341 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:36.341 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:51:36.341 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T11:51:36.341 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:36.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.436 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:51:36.473 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:36.473 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:36.473 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:36.473 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 46:c7:d8:ef:aa:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::44c7:d8ff:feef:aa52/64 scope link 2023-11-28T11:51:36.474 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.485 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:51:36.566 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:51:36.566 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:51:36.566 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 46:c7:d8:ef:aa:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::44c7:d8ff:feef:aa52/64 scope link 2023-11-28T11:51:36.567 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:51:36.587 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T11:51:36.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T11:51:36.588 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:36.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:36.843 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-11-28T11:51:36.843 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-11-28T11:51:36.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.1 up 2023-11-28T11:51:36.843 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:51:36.843 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-11-28T11:51:36.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:36.949 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T11:51:36.968 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:51:36.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:51:36.969 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:51:37.025 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:51:37.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:37.026 DEBUG:teuthology.orchestra.run.smithi177:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:51:37.110 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:51:37.110 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:51:37.110 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T11:51:37.118 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:51:37.119 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:51:37.119 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T11:51:37.119 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:37.119 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:invalid new device string format 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:51:37.145 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:51:37.146 INFO:teuthology.orchestra.run.smithi177.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:51:37.148 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:51:37.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:37.148 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:51:37.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:37.178 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:37.250 INFO:teuthology.orchestra.run.smithi177.stdout:client4481 2023-11-28T11:51:37.250 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:37.250 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:37.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:37.313 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:37.385 INFO:teuthology.orchestra.run.smithi177.stdout:client4481 2023-11-28T11:51:37.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:37.385 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:51:37.457 INFO:teuthology.orchestra.run.smithi177.stdout:client4481 2023-11-28T11:51:37.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:37.457 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/af726c13-c3f1-4279-9fca-12c6e4621b18.client4481/status') 2023-11-28T11:51:37.537 INFO:teuthology.orchestra.run.smithi177.stdout:instance: client.4481 (3)192.168.0.2:0/3322671432 2023-11-28T11:51:37.537 INFO:teuthology.orchestra.run.smithi177.stdout:blocklisted: false 2023-11-28T11:51:37.537 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records in 2023-11-28T11:51:37.537 INFO:teuthology.orchestra.run.smithi177.stderr:0+1 records out 2023-11-28T11:51:37.537 INFO:teuthology.orchestra.run.smithi177.stderr:69 bytes copied, 2.2496e-05 s, 3.1 MB/s 2023-11-28T11:51:37.538 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T11:51:37.560 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:37.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.693+0000 7fea0b14b700 1 Processor -- start 2023-11-28T11:51:37.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.693+0000 7fea0b14b700 1 -- start start 2023-11-28T11:51:37.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea0b14b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fea040d0a00 0x7fea0415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:37.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea0b14b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040b2930 0x7fea040b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:37.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea0b14b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0406b420 con 0x7fea040ce040 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea0b14b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040b32f0 con 0x7fea040d0a00 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea0b14b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0406ff60 con 0x7fea040b2930 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fea040d0a00 0x7fea0415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea08ee7700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fea040ce040 0x7fea0406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51812/0 (socket says 172.21.15.144:51812) 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea08ee7700 1 -- 172.21.15.144:0/2230262538 learned_addr learned my addr 172.21.15.144:0/2230262538 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fea040d0a00 0x7fea0415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35102/0 (socket says 172.21.15.144:35102) 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea096e8700 1 --2- 172.21.15.144:0/2230262538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040b2930 0x7fea040b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 -- 172.21.15.144:0/2230262538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040b2930 msgr2=0x7fea040b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 --2- 172.21.15.144:0/2230262538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040b2930 0x7fea040b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 -- 172.21.15.144:0/2230262538 >> v1:172.21.15.144:6789/0 conn(0x7fea040ce040 legacy=0x7fea0406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea037fe700 1 -- 172.21.15.144:0/2230262538 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790530504 0 0) 0x7fea0406b420 con 0x7fea040ce040 2023-11-28T11:51:37.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 -- 172.21.15.144:0/2230262538 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0406b700 con 0x7fea040d0a00 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.694+0000 7fea03fff700 1 --2- 172.21.15.144:0/2230262538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040d0a00 0x7fea0415df10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009840 tx=0x7fe9f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e32ad6a8bc603 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea037fe700 1 -- 172.21.15.144:0/2230262538 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0011830 con 0x7fea040d0a00 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea037fe700 1 -- 172.21.15.144:0/2230262538 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f0011990 con 0x7fea040d0a00 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea037fe700 1 -- 172.21.15.144:0/2230262538 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0011d20 con 0x7fea040d0a00 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 -- 172.21.15.144:0/2230262538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040d0a00 msgr2=0x7fea0415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 --2- 172.21.15.144:0/2230262538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040d0a00 0x7fea0415df10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009840 tx=0x7fe9f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 -- 172.21.15.144:0/2230262538 shutdown_connections 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 --2- 172.21.15.144:0/2230262538 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040b2930 0x7fea040b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 --2- 172.21.15.144:0/2230262538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040d0a00 0x7fea0415df10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 -- 172.21.15.144:0/2230262538 >> 172.21.15.144:0/2230262538 conn(0x7fea0406aed0 msgr2=0x7fea0406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 -- 172.21.15.144:0/2230262538 shutdown_connections 2023-11-28T11:51:37.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.695+0000 7fea0b14b700 1 -- 172.21.15.144:0/2230262538 wait complete. 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.697+0000 7fea0b14b700 1 Processor -- start 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.697+0000 7fea0b14b700 1 -- start start 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea0b14b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 0x7fea041f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea0b14b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea040ce040 0x7fea041ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea0b14b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040d0a00 0x7fea041f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea0b14b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041f6110 con 0x7fea040b2930 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea0b14b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041f5a70 con 0x7fea040ce040 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea0b14b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea041f2fb0 con 0x7fea040d0a00 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 0x7fea041f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 0x7fea041f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35106/0 (socket says 172.21.15.144:35106) 2023-11-28T11:51:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 -- 172.21.15.144:0/153267208 learned_addr learned my addr 172.21.15.144:0/153267208 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea096e8700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040d0a00 0x7fea041f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea03fff700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea040ce040 0x7fea041ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 -- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040d0a00 msgr2=0x7fea041f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040d0a00 0x7fea041f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 -- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea040ce040 msgr2=0x7fea041ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea040ce040 0x7fea041ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea08ee7700 1 -- 172.21.15.144:0/153267208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00094a0 con 0x7fea040b2930 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea03fff700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea040ce040 0x7fea041ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.698+0000 7fea096e8700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040d0a00 0x7fea041f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.699+0000 7fea08ee7700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 0x7fea041f6530 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80010e0 tx=0x7fe9f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.699+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80074e0 con 0x7fea040b2930 2023-11-28T11:51:37.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.699+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8007660 con 0x7fea040b2930 2023-11-28T11:51:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.699+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0406bcc0 con 0x7fea040b2930 2023-11-28T11:51:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.699+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8005a70 con 0x7fea040b2930 2023-11-28T11:51:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.699+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0406b9e0 con 0x7fea040b2930 2023-11-28T11:51:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.700+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0406bfa0 con 0x7fea040b2930 2023-11-28T11:51:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.700+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe9f8019030 con 0x7fea040b2930 2023-11-28T11:51:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.701+0000 7fea017fa700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9e80682f0 0x7fe9e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.701+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9f8085050 con 0x7fea040b2930 2023-11-28T11:51:37.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.701+0000 7fea03fff700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9e80682f0 0x7fe9e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:37.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.701+0000 7fea03fff700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9e80682f0 0x7fe9e806a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00090b0 tx=0x7fe9f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:37.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.705+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9f8051830 con 0x7fea040b2930 2023-11-28T11:51:37.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.890+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea04068690 con 0x7fea040b2930 2023-11-28T11:51:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.891+0000 7fea017fa700 1 -- 172.21.15.144:0/153267208 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fe9f8017090 con 0x7fea040b2930 2023-11-28T11:51:37.892 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:37.892 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9e80682f0 msgr2=0x7fe9e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9e80682f0 0x7fe9e806a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00090b0 tx=0x7fe9f0008030 comp rx=0 tx=0).stop 2023-11-28T11:51:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 msgr2=0x7fea041f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 0x7fea041f6530 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80010e0 tx=0x7fe9f800c860 comp rx=0 tx=0).stop 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 shutdown_connections 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea040b2930 0x7fea041f6530 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9e80682f0 0x7fe9e806a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea040d0a00 0x7fea041f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 --2- 172.21.15.144:0/153267208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea040ce040 0x7fea041ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 >> 172.21.15.144:0/153267208 conn(0x7fea0406aed0 msgr2=0x7fea040cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.893+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 shutdown_connections 2023-11-28T11:51:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:37.894+0000 7fea0b14b700 1 -- 172.21.15.144:0/153267208 wait complete. 2023-11-28T11:51:37.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:37.907 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:38.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.030+0000 7fd5ea917700 1 Processor -- start 2023-11-28T11:51:38.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.030+0000 7fd5ea917700 1 -- start start 2023-11-28T11:51:38.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.031+0000 7fd5ea917700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e4071040 0x7fd5e4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.031+0000 7fd5ea917700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e4071950 con 0x7fd5e4160540 2023-11-28T11:51:38.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.031+0000 7fd5ea917700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e415d060 con 0x7fd5e406dee0 2023-11-28T11:51:38.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.031+0000 7fd5ea917700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e415d340 con 0x7fd5e4071040 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.031+0000 7fd5e8eb4700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd5e4160540 0x7fd5e41609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51830/0 (socket says 172.21.15.144:51830) 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.031+0000 7fd5e8eb4700 1 -- 172.21.15.144:0/2012358768 learned_addr learned my addr 172.21.15.144:0/2012358768 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e3fff700 1 --2- 172.21.15.144:0/2012358768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e4071040 0x7fd5e4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33137986 0 0) 0x7fd5e4071950 con 0x7fd5e4160540 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5cc0035b0 con 0x7fd5e4160540 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225410815 0 0) 0x7fd5e415d060 con 0x7fd5e406dee0 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e4071950 con 0x7fd5e406dee0 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e3fff700 1 -- 172.21.15.144:0/2012358768 >> v1:172.21.15.177:6789/0 conn(0x7fd5e406dee0 legacy=0x7fd5e406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e3fff700 1 -- 172.21.15.144:0/2012358768 >> v1:172.21.15.144:6789/0 conn(0x7fd5e4160540 legacy=0x7fd5e41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e3fff700 1 -- 172.21.15.144:0/2012358768 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e415d620 con 0x7fd5e4071040 2023-11-28T11:51:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e3fff700 1 --2- 172.21.15.144:0/2012358768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e4071040 0x7fd5e4071410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50cc98260d3d4b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8011830 con 0x7fd5e4071040 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d8011990 con 0x7fd5e4071040 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.032+0000 7fd5e2ffd700 1 -- 172.21.15.144:0/2012358768 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8011d20 con 0x7fd5e4071040 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 -- 172.21.15.144:0/2012358768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e4071040 msgr2=0x7fd5e4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 --2- 172.21.15.144:0/2012358768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e4071040 0x7fd5e4071410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009870 tx=0x7fd5d8009b80 comp rx=0 tx=0).stop 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 -- 172.21.15.144:0/2012358768 shutdown_connections 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 --2- 172.21.15.144:0/2012358768 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e4071040 0x7fd5e4071410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 -- 172.21.15.144:0/2012358768 >> 172.21.15.144:0/2012358768 conn(0x7fd5e40d0000 msgr2=0x7fd5e40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 -- 172.21.15.144:0/2012358768 shutdown_connections 2023-11-28T11:51:38.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.033+0000 7fd5ea917700 1 -- 172.21.15.144:0/2012358768 wait complete. 2023-11-28T11:51:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.034+0000 7fd5ea917700 1 Processor -- start 2023-11-28T11:51:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.034+0000 7fd5ea917700 1 -- start start 2023-11-28T11:51:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5ea917700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e406dee0 0x7fd5e4156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5ea917700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 0x7fd5e40b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5ea917700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e4160540 0x7fd5e40b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5ea917700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40b5dd0 con 0x7fd5e4071040 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5ea917700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40c4680 con 0x7fd5e4160540 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5ea917700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40c4960 con 0x7fd5e406dee0 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 0x7fd5e40b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 0x7fd5e40b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35122/0 (socket says 172.21.15.144:35122) 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 -- 172.21.15.144:0/3934892051 learned_addr learned my addr 172.21.15.144:0/3934892051 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e3fff700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e406dee0 0x7fd5e4156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e8eb4700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e4160540 0x7fd5e40b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 -- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e406dee0 msgr2=0x7fd5e4156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e406dee0 0x7fd5e4156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 -- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e4160540 msgr2=0x7fd5e40b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e4160540 0x7fd5e40b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.035+0000 7fd5e37fe700 1 -- 172.21.15.144:0/3934892051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80094a0 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.036+0000 7fd5e37fe700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 0x7fd5e40b0a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400c500 tx=0x7fd5d400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.036+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40052d0 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.036+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d4005450 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.036+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e415ccd0 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.036+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40038c0 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.036+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e415d900 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.037+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e406a270 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.037+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd5d4019030 con 0x7fd5e4071040 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.038+0000 7fd5e0ff9700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5d00682f0 0x7fd5d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.038+0000 7fd5e3fff700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5d00682f0 0x7fd5d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.038+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5d4084ee0 con 0x7fd5e4071040 2023-11-28T11:51:38.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.038+0000 7fd5e3fff700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5d00682f0 0x7fd5d006a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80097c0 tx=0x7fd5d801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.042+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd5d4051740 con 0x7fd5e4071040 2023-11-28T11:51:38.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.221+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd5e40ccdb0 con 0x7fd5e4071040 2023-11-28T11:51:38.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.222+0000 7fd5e0ff9700 1 -- 172.21.15.144:0/3934892051 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd5d40170a0 con 0x7fd5e4071040 2023-11-28T11:51:38.223 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5d00682f0 msgr2=0x7fd5d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5d00682f0 0x7fd5d006a7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80097c0 tx=0x7fd5d801c040 comp rx=0 tx=0).stop 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 msgr2=0x7fd5e40b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 0x7fd5e40b0a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400c500 tx=0x7fd5d400c810 comp rx=0 tx=0).stop 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 shutdown_connections 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5d00682f0 0x7fd5d006a7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e4160540 0x7fd5e40b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e406dee0 0x7fd5e4156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 --2- 172.21.15.144:0/3934892051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5e4071040 0x7fd5e40b0a50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 >> 172.21.15.144:0/3934892051 conn(0x7fd5e40d0000 msgr2=0x7fd5e415e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:38.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 shutdown_connections 2023-11-28T11:51:38.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.224+0000 7fd5ea917700 1 -- 172.21.15.144:0/3934892051 wait complete. 2023-11-28T11:51:38.238 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:38.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.361+0000 7fe9c9301700 1 Processor -- start 2023-11-28T11:51:38.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.361+0000 7fe9c9301700 1 -- start start 2023-11-28T11:51:38.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.361+0000 7fe9c9301700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9c415ca50 0x7fe9c406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.361+0000 7fe9c9301700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 0x7fe9c40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c9301700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c40cf5e0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c9301700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c40cd470 con 0x7fe9c415ca50 2023-11-28T11:51:38.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c9301700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c406bd30 con 0x7fe9c406b8c0 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c2ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9c415ca50 0x7fe9c406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 0x7fe9c40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c2ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe9c415ca50 0x7fe9c406b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:37402/0 (socket says 172.21.15.144:37402) 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c37fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe9c40ccc40 0x7fe9c40cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:52334/0 (socket says 172.21.15.144:52334) 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 0x7fe9c40cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47210/0 (socket says 172.21.15.144:47210) 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c2ffd700 1 -- 172.21.15.144:0/2373666022 learned_addr learned my addr 172.21.15.144:0/2373666022 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c1ffb700 1 -- 172.21.15.144:0/2373666022 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657499141 0 0) 0x7fe9c40cf5e0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 -- 172.21.15.144:0/2373666022 >> v2:172.21.15.177:3300/0 conn(0x7fe9c415ca50 msgr2=0x7fe9c406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 --2- 172.21.15.144:0/2373666022 >> v2:172.21.15.177:3300/0 conn(0x7fe9c415ca50 0x7fe9c406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 -- 172.21.15.144:0/2373666022 >> v1:172.21.15.177:6789/0 conn(0x7fe9c40ccc40 legacy=0x7fe9c40cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 -- 172.21.15.144:0/2373666022 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40cecb0 con 0x7fe9c406b8c0 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.362+0000 7fe9c27fc700 1 --2- 172.21.15.144:0/2373666022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 0x7fe9c40cc700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009840 tx=0x7fe9b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1548b20e5cd51b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c1ffb700 1 -- 172.21.15.144:0/2373666022 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8011830 con 0x7fe9c406b8c0 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c1ffb700 1 -- 172.21.15.144:0/2373666022 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8011990 con 0x7fe9c406b8c0 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c1ffb700 1 -- 172.21.15.144:0/2373666022 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8011d20 con 0x7fe9c406b8c0 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 -- 172.21.15.144:0/2373666022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 msgr2=0x7fe9c40cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 --2- 172.21.15.144:0/2373666022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 0x7fe9c40cc700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009840 tx=0x7fe9b8009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 -- 172.21.15.144:0/2373666022 shutdown_connections 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 --2- 172.21.15.144:0/2373666022 >> v2:172.21.15.177:3300/0 conn(0x7fe9c415ca50 0x7fe9c406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 --2- 172.21.15.144:0/2373666022 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c406b8c0 0x7fe9c40cc700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 -- 172.21.15.144:0/2373666022 >> 172.21.15.144:0/2373666022 conn(0x7fe9c409a230 msgr2=0x7fe9c408afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 -- 172.21.15.144:0/2373666022 shutdown_connections 2023-11-28T11:51:38.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.363+0000 7fe9c9301700 1 -- 172.21.15.144:0/2373666022 wait complete. 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.365+0000 7fe9c9301700 1 Processor -- start 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.365+0000 7fe9c9301700 1 -- start start 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c9301700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9c406b8c0 0x7fe9c40d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c9301700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 0x7fe9c41f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c9301700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c415ca50 0x7fe9c41f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c9301700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40b4ab0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c9301700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41f20c0 con 0x7fe9c406b8c0 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c9301700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4159110 con 0x7fe9c415ca50 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 0x7fe9c41f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 0x7fe9c41f1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35136/0 (socket says 172.21.15.144:35136) 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 -- 172.21.15.144:0/1772752443 learned_addr learned my addr 172.21.15.144:0/1772752443 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c37fe700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c415ca50 0x7fe9c41f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c2ffd700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9c406b8c0 0x7fe9c40d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 -- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c415ca50 msgr2=0x7fe9c41f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c415ca50 0x7fe9c41f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 -- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9c406b8c0 msgr2=0x7fe9c40d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9c406b8c0 0x7fe9c40d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 -- 172.21.15.144:0/1772752443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80094a0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c37fe700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c415ca50 0x7fe9c41f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c2ffd700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9c406b8c0 0x7fe9c40d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.366+0000 7fe9c27fc700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 0x7fe9c41f1640 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009fd0 tx=0x7fe9b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.367+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80036a0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.367+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b801f070 con 0x7fe9c40ccc40 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.367+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c40cf270 con 0x7fe9c40ccc40 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.367+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003a40 con 0x7fe9c40ccc40 2023-11-28T11:51:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.367+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c40cef90 con 0x7fe9c40ccc40 2023-11-28T11:51:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.368+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c40cd0c0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.368+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe9b800f050 con 0x7fe9c40ccc40 2023-11-28T11:51:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.368+0000 7fe9ab7fe700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9b00682f0 0x7fe9b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.368+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9b8021070 con 0x7fe9c40ccc40 2023-11-28T11:51:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.368+0000 7fe9c2ffd700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9b00682f0 0x7fe9b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.369+0000 7fe9c2ffd700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9b00682f0 0x7fe9b006a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009ab0 tx=0x7fe9b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.373+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9b8053e80 con 0x7fe9c40ccc40 2023-11-28T11:51:38.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.553+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9c40b4bf0 con 0x7fe9c40ccc40 2023-11-28T11:51:38.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.553+0000 7fe9ab7fe700 1 -- 172.21.15.144:0/1772752443 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9b8015020 con 0x7fe9c40ccc40 2023-11-28T11:51:38.554 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:38.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9b00682f0 msgr2=0x7fe9b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9b00682f0 0x7fe9b006a7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009ab0 tx=0x7fe9b4008040 comp rx=0 tx=0).stop 2023-11-28T11:51:38.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 msgr2=0x7fe9c41f1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 0x7fe9c41f1640 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009fd0 tx=0x7fe9b8004750 comp rx=0 tx=0).stop 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 shutdown_connections 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9b00682f0 0x7fe9b006a7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe9c415ca50 0x7fe9c41f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9c406b8c0 0x7fe9c40d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 --2- 172.21.15.144:0/1772752443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c40ccc40 0x7fe9c41f1640 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 >> 172.21.15.144:0/1772752443 conn(0x7fe9c409a230 msgr2=0x7fe9c406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 shutdown_connections 2023-11-28T11:51:38.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.555+0000 7fe9c9301700 1 -- 172.21.15.144:0/1772752443 wait complete. 2023-11-28T11:51:38.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:38.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.691+0000 7fbccfdf8700 1 Processor -- start 2023-11-28T11:51:38.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.691+0000 7fbccfdf8700 1 -- start start 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccfdf8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 0x7fbcc8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccfdf8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 0x7fbcc81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccfdf8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc8071950 con 0x7fbcc806dee0 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccfdf8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc815d060 con 0x7fbcc8071040 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccfdf8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc815d340 con 0x7fbcc8160540 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbcce395700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 0x7fbcc81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccdb94700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 0x7fbcc8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbcce395700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 0x7fbcc81609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47226/0 (socket says 172.21.15.144:47226) 2023-11-28T11:51:38.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbcce395700 1 -- 172.21.15.144:0/3885140594 learned_addr learned my addr 172.21.15.144:0/3885140594 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccdb94700 1 --2- 172.21.15.144:0/3885140594 >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 0x7fbcc8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:37416/0 (socket says 172.21.15.144:37416) 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.692+0000 7fbccd393700 1 --1- 172.21.15.144:0/3885140594 >> v1:172.21.15.177:6789/0 conn(0x7fbcc806dee0 0x7fbcc806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:52344/0 (socket says 172.21.15.144:52344) 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcccb92700 1 -- 172.21.15.144:0/3885140594 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595834085 0 0) 0x7fbcc8071950 con 0x7fbcc806dee0 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcccb92700 1 -- 172.21.15.144:0/3885140594 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb40035b0 con 0x7fbcc806dee0 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcce395700 1 -- 172.21.15.144:0/3885140594 >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 msgr2=0x7fbcc8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcce395700 1 --2- 172.21.15.144:0/3885140594 >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 0x7fbcc8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcce395700 1 -- 172.21.15.144:0/3885140594 >> v1:172.21.15.177:6789/0 conn(0x7fbcc806dee0 legacy=0x7fbcc806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcce395700 1 -- 172.21.15.144:0/3885140594 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc815d620 con 0x7fbcc8160540 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbccdb94700 1 --2- 172.21.15.144:0/3885140594 >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 0x7fbcc8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcce395700 1 --2- 172.21.15.144:0/3885140594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 0x7fbcc81609c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e515b607d47031c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcccb92700 1 -- 172.21.15.144:0/3885140594 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000e070 con 0x7fbcc8160540 2023-11-28T11:51:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcccb92700 1 -- 172.21.15.144:0/3885140594 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc000b910 con 0x7fbcc8160540 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.693+0000 7fbcccb92700 1 -- 172.21.15.144:0/3885140594 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000bcc0 con 0x7fbcc8160540 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 -- 172.21.15.144:0/3885140594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 msgr2=0x7fbcc81609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/3885140594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 0x7fbcc81609c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 -- 172.21.15.144:0/3885140594 shutdown_connections 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/3885140594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8160540 0x7fbcc81609c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/3885140594 >> v2:172.21.15.177:3300/0 conn(0x7fbcc8071040 0x7fbcc8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 -- 172.21.15.144:0/3885140594 >> 172.21.15.144:0/3885140594 conn(0x7fbcc80d0000 msgr2=0x7fbcc80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 -- 172.21.15.144:0/3885140594 shutdown_connections 2023-11-28T11:51:38.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.694+0000 7fbccfdf8700 1 -- 172.21.15.144:0/3885140594 wait complete. 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.695+0000 7fbccfdf8700 1 Processor -- start 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.695+0000 7fbccfdf8700 1 -- start start 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccfdf8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc806dee0 0x7fbcc8157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccfdf8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8071040 0x7fbcc80aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccfdf8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 0x7fbcc80af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccfdf8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81f34c0 con 0x7fbcc8160540 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccfdf8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81577b0 con 0x7fbcc806dee0 2023-11-28T11:51:38.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccfdf8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc81f2880 con 0x7fbcc8071040 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 0x7fbcc80af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 0x7fbcc80af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35150/0 (socket says 172.21.15.144:35150) 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 -- 172.21.15.144:0/4275026274 learned_addr learned my addr 172.21.15.144:0/4275026274 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccd393700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8071040 0x7fbcc80aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccdb94700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc806dee0 0x7fbcc8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 -- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8071040 msgr2=0x7fbcc80aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8071040 0x7fbcc80aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 -- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc806dee0 msgr2=0x7fbcc8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc806dee0 0x7fbcc8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 -- 172.21.15.144:0/4275026274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80094a0 con 0x7fbcc8160540 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbccdb94700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc806dee0 0x7fbcc8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:38.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcce395700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 0x7fbcc80af300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcc815d480 tx=0x7fbcc0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0004960 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.696+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc00099b0 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.697+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc0009b30 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.697+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc001c680 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.697+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc815ccd0 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.697+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc815d900 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.698+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbcc002c070 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.699+0000 7fbcbe7fc700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbcac0682f0 0x7fbcac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.699+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbcc0087ce0 con 0x7fbcc8160540 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.699+0000 7fbccdb94700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbcac0682f0 0x7fbcac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:38.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.699+0000 7fbccdb94700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbcac0682f0 0x7fbcac06a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009c50 tx=0x7fbcb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:38.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.704+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbcc00545e0 con 0x7fbcc8160540 2023-11-28T11:51:38.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.886+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcc806a270 con 0x7fbcc8160540 2023-11-28T11:51:38.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.887+0000 7fbcbe7fc700 1 -- 172.21.15.144:0/4275026274 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbcc0057c00 con 0x7fbcc8160540 2023-11-28T11:51:38.888 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.888+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbcac0682f0 msgr2=0x7fbcac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.888+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbcac0682f0 0x7fbcac06a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009c50 tx=0x7fbcb8008040 comp rx=0 tx=0).stop 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 msgr2=0x7fbcc80af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 0x7fbcc80af300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcc815d480 tx=0x7fbcc0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 shutdown_connections 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc8160540 0x7fbcc80af300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbcac0682f0 0x7fbcac06a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc806dee0 0x7fbcc8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 --2- 172.21.15.144:0/4275026274 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc8071040 0x7fbcc80aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:38.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 >> 172.21.15.144:0/4275026274 conn(0x7fbcc80d0000 msgr2=0x7fbcc815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:38.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 shutdown_connections 2023-11-28T11:51:38.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:38.889+0000 7fbccfdf8700 1 -- 172.21.15.144:0/4275026274 wait complete. 2023-11-28T11:51:38.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:39.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.023+0000 7f778b176700 1 Processor -- start 2023-11-28T11:51:39.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.023+0000 7f778b176700 1 -- start start 2023-11-28T11:51:39.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.024+0000 7f778b176700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f778406e500 0x7f778415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.024+0000 7f778b176700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840aee40 con 0x7f778406a5e0 2023-11-28T11:51:39.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.024+0000 7f778b176700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778406c930 con 0x7f77840ae9d0 2023-11-28T11:51:39.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.024+0000 7f778b176700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778415bf10 con 0x7f778406e500 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.024+0000 7f7789713700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f778406a5e0 0x7f778406a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51870/0 (socket says 172.21.15.144:51870) 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.024+0000 7f7789713700 1 -- 172.21.15.144:0/3366227897 learned_addr learned my addr 172.21.15.144:0/3366227897 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448335097 0 0) 0x7f77840aee40 con 0x7f778406a5e0 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77740035b0 con 0x7f778406a5e0 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7788f12700 1 --2- 172.21.15.144:0/3366227897 >> v2:172.21.15.177:3300/0 conn(0x7f778406e500 0x7f778415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2970617847 0 0) 0x7f77740035b0 con 0x7f778406a5e0 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 >> v2:172.21.15.177:3300/0 conn(0x7f778406e500 msgr2=0x7f778415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 --2- 172.21.15.144:0/3366227897 >> v2:172.21.15.177:3300/0 conn(0x7f778406e500 0x7f778415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 >> v1:172.21.15.177:6789/0 conn(0x7f77840ae9d0 legacy=0x7f778406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778406cc10 con 0x7f778406a5e0 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f777c0019f0 con 0x7f778406a5e0 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7788f12700 1 --2- 172.21.15.144:0/3366227897 >> v2:172.21.15.177:3300/0 conn(0x7f778406e500 0x7f778415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 >> v1:172.21.15.144:6789/0 conn(0x7f778406a5e0 legacy=0x7f778406a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774003a00 0x7f7774003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7774004850 0x7f7774004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7774005400 0x7f777400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778415bf10 con 0x7f7774005400 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777400c280 con 0x7f7774003a00 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777400c440 con 0x7f7774004850 2023-11-28T11:51:39.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7774005400 0x7f777400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7788f12700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774003a00 0x7f7774003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7783fff700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7774004850 0x7f7774004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 -- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7774004850 msgr2=0x7f7774004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7774004850 0x7f7774004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 -- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774003a00 msgr2=0x7f7774003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774003a00 0x7f7774003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 -- 172.21.15.144:0/3366227897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740035b0 con 0x7f7774005400 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.025+0000 7f7789713700 1 -- 172.21.15.144:0/3366227897 reap_dead start 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f7789713700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7774005400 0x7f777400ba10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f777c00a830 tx=0x7f7774005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ea5c1e76e2e88f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c00a2f0 con 0x7f7774005400 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c012620 con 0x7f7774005400 2023-11-28T11:51:39.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f77837fe700 1 -- 172.21.15.144:0/3366227897 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0129d0 con 0x7f7774005400 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 -- 172.21.15.144:0/3366227897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7774005400 msgr2=0x7f777400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7774005400 0x7f777400ba10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f777c00a830 tx=0x7f7774005880 comp rx=0 tx=0).stop 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 -- 172.21.15.144:0/3366227897 shutdown_connections 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 --2- 172.21.15.144:0/3366227897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7774005400 0x7f777400ba10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 -- 172.21.15.144:0/3366227897 >> 172.21.15.144:0/3366227897 conn(0x7f77840b12b0 msgr2=0x7f77840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 -- 172.21.15.144:0/3366227897 shutdown_connections 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.026+0000 7f778b176700 1 -- 172.21.15.144:0/3366227897 wait complete. 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.027+0000 7f778b176700 1 Processor -- start 2023-11-28T11:51:39.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.027+0000 7f778b176700 1 -- start start 2023-11-28T11:51:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f778b176700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774005400 0x7f7784144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f778b176700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 0x7f77840b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f778b176700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77840b3510 0x7f77840c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f778b176700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778415d130 con 0x7f778414eda0 2023-11-28T11:51:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f778b176700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7784145080 con 0x7f7774005400 2023-11-28T11:51:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f778b176700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77840b3fa0 con 0x7f77840b3510 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f7783fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 0x7f77840b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f7783fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 0x7f77840b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35168/0 (socket says 172.21.15.144:35168) 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.028+0000 7f7783fff700 1 -- 172.21.15.144:0/249364972 learned_addr learned my addr 172.21.15.144:0/249364972 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7788f12700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774005400 0x7f7784144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7789713700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77840b3510 0x7f77840c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7783fff700 1 -- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77840b3510 msgr2=0x7f77840c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7783fff700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77840b3510 0x7f77840c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7783fff700 1 -- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774005400 msgr2=0x7f7784144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7783fff700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774005400 0x7f7784144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7783fff700 1 -- 172.21.15.144:0/249364972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778406cef0 con 0x7f778414eda0 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7788f12700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774005400 0x7f7784144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f7783fff700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 0x7f77840b2fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f777000e490 tx=0x7f777000e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77700052e0 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770005440 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77700038c0 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77840664c0 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.029+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778415ac80 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.030+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7784065fc0 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.030+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f777001b030 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.031+0000 7f77817fa700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77680682f0 0x7f776806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.031+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7770085160 con 0x7f778414eda0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.031+0000 7f7788f12700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77680682f0 0x7f776806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.031+0000 7f7788f12700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77680682f0 0x7f776806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7778004510 tx=0x7f7778008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.035+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f777004d5e0 con 0x7f778414eda0 2023-11-28T11:51:39.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.217+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77841595e0 con 0x7f778414eda0 2023-11-28T11:51:39.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.217+0000 7f77817fa700 1 -- 172.21.15.144:0/249364972 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7770019020 con 0x7f778414eda0 2023-11-28T11:51:39.218 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77680682f0 msgr2=0x7f776806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77680682f0 0x7f776806a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7778004510 tx=0x7f7778008040 comp rx=0 tx=0).stop 2023-11-28T11:51:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 msgr2=0x7f77840b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 0x7f77840b2fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f777000e490 tx=0x7f777000e850 comp rx=0 tx=0).stop 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 shutdown_connections 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77680682f0 0x7f776806a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77840b3510 0x7f77840c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7774005400 0x7f7784144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 --2- 172.21.15.144:0/249364972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f778414eda0 0x7f77840b2fd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 >> 172.21.15.144:0/249364972 conn(0x7f77840b12b0 msgr2=0x7f77840af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.219+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 shutdown_connections 2023-11-28T11:51:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.220+0000 7f778b176700 1 -- 172.21.15.144:0/249364972 wait complete. 2023-11-28T11:51:39.233 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:39.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.351+0000 7f6226b6d700 1 Processor -- start 2023-11-28T11:51:39.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.351+0000 7f6226b6d700 1 -- start start 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6226b6d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6220071040 0x7f6220071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6226b6d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f622006dee0 0x7f622006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6226b6d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6220071950 con 0x7f622006cdc0 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6226b6d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6220161550 con 0x7f6220071040 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6226b6d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622015d060 con 0x7f622006dee0 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6224909700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6220071040 0x7f6220071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6224909700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6220071040 0x7f6220071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35174/0 (socket says 172.21.15.144:35174) 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f6224909700 1 -- 172.21.15.144:0/4239703397 learned_addr learned my addr 172.21.15.144:0/4239703397 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.352+0000 7f621ffff700 1 --2- 172.21.15.144:0/4239703397 >> v2:172.21.15.177:3300/0 conn(0x7f622006dee0 0x7f622006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6224909700 1 -- 172.21.15.144:0/4239703397 >> v2:172.21.15.177:3300/0 conn(0x7f622006dee0 msgr2=0x7f622006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:39.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6224909700 1 --2- 172.21.15.144:0/4239703397 >> v2:172.21.15.177:3300/0 conn(0x7f622006dee0 0x7f622006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6224909700 1 -- 172.21.15.144:0/4239703397 >> v1:172.21.15.177:6789/0 conn(0x7f622006cdc0 legacy=0x7f622006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6224909700 1 -- 172.21.15.144:0/4239703397 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622015d340 con 0x7f6220071040 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6224909700 1 --2- 172.21.15.144:0/4239703397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f6220071410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f622006fb00 tx=0x7f6214009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f12f3aaae49d62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f621f7fe700 1 -- 172.21.15.144:0/4239703397 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214004230 con 0x7f6220071040 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f621f7fe700 1 -- 172.21.15.144:0/4239703397 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6214004390 con 0x7f6220071040 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f621f7fe700 1 -- 172.21.15.144:0/4239703397 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62140036a0 con 0x7f6220071040 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 -- 172.21.15.144:0/4239703397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 msgr2=0x7f6220071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 --2- 172.21.15.144:0/4239703397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f6220071410 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f622006fb00 tx=0x7f6214009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 -- 172.21.15.144:0/4239703397 shutdown_connections 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 --2- 172.21.15.144:0/4239703397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f6220071410 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 --2- 172.21.15.144:0/4239703397 >> v2:172.21.15.177:3300/0 conn(0x7f622006dee0 0x7f622006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 -- 172.21.15.144:0/4239703397 >> 172.21.15.144:0/4239703397 conn(0x7f62200d0000 msgr2=0x7f62200d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:39.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.353+0000 7f6226b6d700 1 -- 172.21.15.144:0/4239703397 shutdown_connections 2023-11-28T11:51:39.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.354+0000 7f6226b6d700 1 -- 172.21.15.144:0/4239703397 wait complete. 2023-11-28T11:51:39.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.355+0000 7f6226b6d700 1 Processor -- start 2023-11-28T11:51:39.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.355+0000 7f6226b6d700 1 -- start start 2023-11-28T11:51:39.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.355+0000 7f6226b6d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f622006cdc0 0x7f62200b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f622006dee0 0x7f6220154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f62200adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62201546b0 con 0x7f6220071040 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62200ae180 con 0x7f622006cdc0 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62200b3480 con 0x7f622006dee0 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f62200adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f62200adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35184/0 (socket says 172.21.15.144:35184) 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 -- 172.21.15.144:0/9752658 learned_addr learned my addr 172.21.15.144:0/9752658 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6224909700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f622006cdc0 0x7f62200b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f621ffff700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f622006dee0 0x7f6220154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 -- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f622006dee0 msgr2=0x7f6220154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f622006dee0 0x7f6220154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 -- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f622006cdc0 msgr2=0x7f62200b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f622006cdc0 0x7f62200b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 -- 172.21.15.144:0/9752658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6214009550 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f621ffff700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f622006dee0 0x7f6220154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f622510a700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f62200adc40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f621800b970 tx=0x7f621800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800c930 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621800cad0 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622015d900 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800ce80 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.356+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622015d620 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.357+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622006e870 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.358+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f621801b030 con 0x7f6220071040 2023-11-28T11:51:39.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.358+0000 7f621d7fa700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6204068320 0x7f620406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.358+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62180856d0 con 0x7f6220071040 2023-11-28T11:51:39.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.358+0000 7f6224909700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6204068320 0x7f620406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.358+0000 7f6224909700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6204068320 0x7f620406a7d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6214009130 tx=0x7f6214008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.362+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f621804d5e0 con 0x7f6220071040 2023-11-28T11:51:39.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.547+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f622009df00 con 0x7f6220071040 2023-11-28T11:51:39.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.548+0000 7f621d7fa700 1 -- 172.21.15.144:0/9752658 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f6218051f80 con 0x7f6220071040 2023-11-28T11:51:39.549 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:39.549 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:39.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6204068320 msgr2=0x7f620406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6204068320 0x7f620406a7d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6214009130 tx=0x7f6214008030 comp rx=0 tx=0).stop 2023-11-28T11:51:39.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 msgr2=0x7f62200adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f62200adc40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f621800b970 tx=0x7f621800bc80 comp rx=0 tx=0).stop 2023-11-28T11:51:39.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 shutdown_connections 2023-11-28T11:51:39.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6220071040 0x7f62200adc40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6204068320 0x7f620406a7d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f622006cdc0 0x7f62200b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 --2- 172.21.15.144:0/9752658 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f622006dee0 0x7f6220154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 >> 172.21.15.144:0/9752658 conn(0x7f62200d0000 msgr2=0x7f622006d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:39.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 shutdown_connections 2023-11-28T11:51:39.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.550+0000 7f6226b6d700 1 -- 172.21.15.144:0/9752658 wait complete. 2023-11-28T11:51:39.564 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:39.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:39.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.688+0000 7f197f91b700 1 Processor -- start 2023-11-28T11:51:39.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.688+0000 7f197f91b700 1 -- start start 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197f91b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1978160740 0x7f19780cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197f91b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1978160b10 0x7f19780cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197f91b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197806bbb0 con 0x7f197806b060 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197f91b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19780ca720 con 0x7f1978160740 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197f91b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19780cc030 con 0x7f1978160b10 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1978160740 0x7f19780cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1978160740 0x7f19780cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35188/0 (socket says 172.21.15.144:35188) 2023-11-28T11:51:39.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 -- 172.21.15.144:0/270138391 learned_addr learned my addr 172.21.15.144:0/270138391 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197ceb6700 1 --2- 172.21.15.144:0/270138391 >> v2:172.21.15.177:3300/0 conn(0x7f1978160b10 0x7f19780cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 -- 172.21.15.144:0/270138391 >> v2:172.21.15.177:3300/0 conn(0x7f1978160b10 msgr2=0x7f19780cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 --2- 172.21.15.144:0/270138391 >> v2:172.21.15.177:3300/0 conn(0x7f1978160b10 0x7f19780cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 -- 172.21.15.144:0/270138391 >> v1:172.21.15.177:6789/0 conn(0x7f197806b060 legacy=0x7f197806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 -- 172.21.15.144:0/270138391 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19780c9e10 con 0x7f1978160740 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197ceb6700 1 --2- 172.21.15.144:0/270138391 >> v2:172.21.15.177:3300/0 conn(0x7f1978160b10 0x7f19780cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f197d6b7700 1 --2- 172.21.15.144:0/270138391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160740 0x7f19780cbaf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19780aeb10 tx=0x7f1974009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04228796dd9fa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f196ffff700 1 -- 172.21.15.144:0/270138391 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400b870 con 0x7f1978160740 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f196ffff700 1 -- 172.21.15.144:0/270138391 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400b9d0 con 0x7f1978160740 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.689+0000 7f196ffff700 1 -- 172.21.15.144:0/270138391 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400bd60 con 0x7f1978160740 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 -- 172.21.15.144:0/270138391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160740 msgr2=0x7f19780cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 --2- 172.21.15.144:0/270138391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160740 0x7f19780cbaf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19780aeb10 tx=0x7f1974009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 -- 172.21.15.144:0/270138391 shutdown_connections 2023-11-28T11:51:39.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 --2- 172.21.15.144:0/270138391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160740 0x7f19780cbaf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 --2- 172.21.15.144:0/270138391 >> v2:172.21.15.177:3300/0 conn(0x7f1978160b10 0x7f19780cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 -- 172.21.15.144:0/270138391 >> 172.21.15.144:0/270138391 conn(0x7f19780b12b0 msgr2=0x7f19780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:39.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 -- 172.21.15.144:0/270138391 shutdown_connections 2023-11-28T11:51:39.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.690+0000 7f197f91b700 1 -- 172.21.15.144:0/270138391 wait complete. 2023-11-28T11:51:39.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.692+0000 7f197f91b700 1 Processor -- start 2023-11-28T11:51:39.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.692+0000 7f197f91b700 1 -- start start 2023-11-28T11:51:39.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.692+0000 7f197f91b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197806b060 0x7f197815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1978160740 0x7f1978150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 0x7f19780c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c6890 con 0x7f1978160b10 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19780c6e10 con 0x7f197806b060 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197815f240 con 0x7f1978160740 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 0x7f19780c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 0x7f19780c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35198/0 (socket says 172.21.15.144:35198) 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 -- 172.21.15.144:0/886373113 learned_addr learned my addr 172.21.15.144:0/886373113 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197ceb6700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1978160740 0x7f1978150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197d6b7700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197806b060 0x7f197815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 -- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1978160740 msgr2=0x7f1978150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1978160740 0x7f1978150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 -- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197806b060 msgr2=0x7f197815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197806b060 0x7f197815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 -- 172.21.15.144:0/886373113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009550 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197deb8700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 0x7f19780c7fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f197000cb50 tx=0x7f197000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970003c00 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1970003d60 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000ab70 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19780ca0d0 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.693+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197806efc0 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.694+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19780ca3b0 con 0x7f1978160b10 2023-11-28T11:51:39.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.695+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1970010050 con 0x7f1978160b10 2023-11-28T11:51:39.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.695+0000 7f196dffb700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f195c0682f0 0x7f195c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:39.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.695+0000 7f197d6b7700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f195c0682f0 0x7f195c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:39.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.695+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1970085b50 con 0x7f1978160b10 2023-11-28T11:51:39.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.695+0000 7f197d6b7700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f195c0682f0 0x7f195c06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1974000c00 tx=0x7f1974008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:39.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.699+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19700523b0 con 0x7f1978160b10 2023-11-28T11:51:39.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.881+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f19780c6f80 con 0x7f1978160b10 2023-11-28T11:51:39.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.881+0000 7f196dffb700 1 -- 172.21.15.144:0/886373113 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f19700559d0 con 0x7f1978160b10 2023-11-28T11:51:39.882 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:39.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.883+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f195c0682f0 msgr2=0x7f195c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.883+0000 7f197f91b700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f195c0682f0 0x7f195c06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1974000c00 tx=0x7f1974008030 comp rx=0 tx=0).stop 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.883+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 msgr2=0x7f19780c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.883+0000 7f197f91b700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 0x7f19780c7fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f197000cb50 tx=0x7f197000cf10 comp rx=0 tx=0).stop 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 shutdown_connections 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1978160b10 0x7f19780c7fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f195c0682f0 0x7f195c06a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197806b060 0x7f197815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 --2- 172.21.15.144:0/886373113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1978160740 0x7f1978150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 >> 172.21.15.144:0/886373113 conn(0x7f19780b12b0 msgr2=0x7f19780ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 shutdown_connections 2023-11-28T11:51:39.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:39.884+0000 7f197f91b700 1 -- 172.21.15.144:0/886373113 wait complete. 2023-11-28T11:51:39.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:40.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.021+0000 7f3da629f700 1 Processor -- start 2023-11-28T11:51:40.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.021+0000 7f3da629f700 1 -- start start 2023-11-28T11:51:40.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3da629f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 0x7f3da006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3da629f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3da0160540 0x7f3da01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3da629f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da015d060 con 0x7f3da0071040 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3da629f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da015d340 con 0x7f3da0160540 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3da629f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015d620 con 0x7f3da006dee0 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9ffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3da0071040 0x7f3da0071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51914/0 (socket says 172.21.15.144:51914) 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9ffff700 1 -- 172.21.15.144:0/3237593605 learned_addr learned my addr 172.21.15.144:0/3237593605 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3da483c700 1 --2- 172.21.15.144:0/3237593605 >> v2:172.21.15.177:3300/0 conn(0x7f3da0160540 0x7f3da01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9f7fe700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 0x7f3da006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389101010 0 0) 0x7f3da015d060 con 0x7f3da0071040 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d880035b0 con 0x7f3da0071040 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187683866 0 0) 0x7f3d880035b0 con 0x7f3da0071040 2023-11-28T11:51:40.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 msgr2=0x7f3da006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 0x7f3da006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 >> v2:172.21.15.177:3300/0 conn(0x7f3da0160540 msgr2=0x7f3da01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 --2- 172.21.15.144:0/3237593605 >> v2:172.21.15.177:3300/0 conn(0x7f3da0160540 0x7f3da01609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da015d900 con 0x7f3da0071040 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f3d94001670 con 0x7f3da0071040 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 >> v1:172.21.15.144:6789/0 conn(0x7f3da0071040 legacy=0x7f3da0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d88003a00 0x7f3d88003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 0x7f3d88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.022+0000 7f3d9effd700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015d340 con 0x7f3d88004d90 2023-11-28T11:51:40.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015d620 con 0x7f3d88003a00 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8800ba90 con 0x7f3da006dee0 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9ffff700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d88003a00 0x7f3d88003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9f7fe700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 0x7f3d88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 -- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 msgr2=0x7f3d88004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da006dee0 0x7f3d88004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 -- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d88003a00 msgr2=0x7f3d88003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d88003a00 0x7f3d88003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 -- 172.21.15.144:0/3237593605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880035b0 con 0x7f3d88004d90 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3da483c700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fc0 tx=0x7f3d88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e6ced0728e89f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800ec30 con 0x7f3d88004d90 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9800f040 con 0x7f3d88004d90 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.023+0000 7f3d9effd700 1 -- 172.21.15.144:0/3237593605 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98014480 con 0x7f3d88004d90 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da629f700 1 -- 172.21.15.144:0/3237593605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d88004d90 msgr2=0x7f3d88005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da629f700 1 --2- 172.21.15.144:0/3237593605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d88004d90 0x7f3d88005200 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fc0 tx=0x7f3d88005880 comp rx=0 tx=0).stop 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da483c700 1 -- 172.21.15.144:0/3237593605 reap_dead start 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da629f700 1 -- 172.21.15.144:0/3237593605 shutdown_connections 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da629f700 1 -- 172.21.15.144:0/3237593605 >> 172.21.15.144:0/3237593605 conn(0x7f3da00d0000 msgr2=0x7f3da00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da629f700 1 -- 172.21.15.144:0/3237593605 shutdown_connections 2023-11-28T11:51:40.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.024+0000 7f3da629f700 1 -- 172.21.15.144:0/3237593605 wait complete. 2023-11-28T11:51:40.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.025+0000 7f3da629f700 1 Processor -- start 2023-11-28T11:51:40.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.025+0000 7f3da629f700 1 -- start start 2023-11-28T11:51:40.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da629f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 0x7f3da014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da629f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da014e060 0x7f3da00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da629f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3da00af340 0x7f3da014bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da629f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da014d700 con 0x7f3da014b450 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da629f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00af7c0 con 0x7f3da00af340 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da629f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da014e7c0 con 0x7f3da014e060 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 0x7f3da014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 0x7f3da014db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35214/0 (socket says 172.21.15.144:35214) 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 -- 172.21.15.144:0/1098674887 learned_addr learned my addr 172.21.15.144:0/1098674887 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9f7fe700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da014e060 0x7f3da00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da483c700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3da00af340 0x7f3da014bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 -- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da014e060 msgr2=0x7f3da00aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da014e060 0x7f3da00aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 -- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3da00af340 msgr2=0x7f3da014bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3da00af340 0x7f3da014bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 -- 172.21.15.144:0/1098674887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da006a270 con 0x7f3da014b450 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3da483c700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3da00af340 0x7f3da014bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.026+0000 7f3d9ffff700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 0x7f3da014db20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d94007310 tx=0x7f3d940076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.027+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940077f0 con 0x7f3da014b450 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.027+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9400f8d0 con 0x7f3da014b450 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.027+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da015ccd0 con 0x7f3da014b450 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.027+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400fca0 con 0x7f3da014b450 2023-11-28T11:51:40.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.027+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da014cd60 con 0x7f3da014b450 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.028+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da006a400 con 0x7f3da014b450 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.028+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d9401d460 con 0x7f3da014b450 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.028+0000 7f3d9cff9700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3d8c0682f0 0x7f3d8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.028+0000 7f3d9f7fe700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3d8c0682f0 0x7f3d8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.029+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d94085e00 con 0x7f3da014b450 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.029+0000 7f3d9f7fe700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3d8c0682f0 0x7f3d8c06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3da00b2ff0 tx=0x7f3d90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.033+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3da006a400 con 0x7f3da014b450 2023-11-28T11:51:40.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.214+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3da006a400 con 0x7f3da014b450 2023-11-28T11:51:40.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.215+0000 7f3d9cff9700 1 -- 172.21.15.144:0/1098674887 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3da006a400 con 0x7f3da014b450 2023-11-28T11:51:40.215 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:40.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.216+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3d8c0682f0 msgr2=0x7f3d8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.216+0000 7f3da629f700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3d8c0682f0 0x7f3d8c06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3da00b2ff0 tx=0x7f3d90008040 comp rx=0 tx=0).stop 2023-11-28T11:51:40.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 msgr2=0x7f3da014db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 0x7f3da014db20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d94007310 tx=0x7f3d940076d0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 shutdown_connections 2023-11-28T11:51:40.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3da014b450 0x7f3da014db20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3d8c0682f0 0x7f3d8c06a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3da00af340 0x7f3da014bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 --2- 172.21.15.144:0/1098674887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3da014e060 0x7f3da00aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 >> 172.21.15.144:0/1098674887 conn(0x7f3da00d0000 msgr2=0x7f3da015f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:40.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 shutdown_connections 2023-11-28T11:51:40.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.217+0000 7f3da629f700 1 -- 172.21.15.144:0/1098674887 wait complete. 2023-11-28T11:51:40.229 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:40.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.352+0000 7fc2d4c41700 1 Processor -- start 2023-11-28T11:51:40.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.353+0000 7fc2d4c41700 1 -- start start 2023-11-28T11:51:40.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.353+0000 7fc2d4c41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 0x7fc2d015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.353+0000 7fc2d4c41700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc2d00b2930 0x7fc2d00b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2d4c41700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d006b420 con 0x7fc2d00d0a00 2023-11-28T11:51:40.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2d4c41700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d00b32f0 con 0x7fc2d00b2930 2023-11-28T11:51:40.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2d4c41700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d006ff60 con 0x7fc2d00ce040 2023-11-28T11:51:40.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cdd9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc2d00d0a00 0x7fc2d006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51936/0 (socket says 172.21.15.144:51936) 2023-11-28T11:51:40.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cdd9b700 1 -- 172.21.15.144:0/4240639137 learned_addr learned my addr 172.21.15.144:0/4240639137 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:40.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2ce59c700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 0x7fc2d015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2ced9d700 1 --2- 172.21.15.144:0/4240639137 >> v2:172.21.15.177:3300/0 conn(0x7fc2d00b2930 0x7fc2d00b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106008071 0 0) 0x7fc2d006b420 con 0x7fc2d00d0a00 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b80035b0 con 0x7fc2d00d0a00 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1575914776 0 0) 0x7fc2b80035b0 con 0x7fc2d00d0a00 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 msgr2=0x7fc2d015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 0x7fc2d015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 >> v2:172.21.15.177:3300/0 conn(0x7fc2d00b2930 msgr2=0x7fc2d00b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 --2- 172.21.15.144:0/4240639137 >> v2:172.21.15.177:3300/0 conn(0x7fc2d00b2930 0x7fc2d00b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d006b700 con 0x7fc2d00d0a00 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fc2c0001670 con 0x7fc2d00d0a00 2023-11-28T11:51:40.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 >> v1:172.21.15.144:6789/0 conn(0x7fc2d00d0a00 legacy=0x7fc2d006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 0x7fc2b8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2b8004d90 0x7fc2b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00b32f0 con 0x7fc2b8003f80 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d006ff60 con 0x7fc2b8004d90 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b800ba10 con 0x7fc2d00ce040 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2ce59c700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 0x7fc2b8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.354+0000 7fc2ced9d700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2b8004d90 0x7fc2b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cdd9b700 1 -- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 msgr2=0x7fc2b8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d00ce040 0x7fc2b8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cdd9b700 1 -- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2b8004d90 msgr2=0x7fc2b8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2b8004d90 0x7fc2b8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cdd9b700 1 -- 172.21.15.144:0/4240639137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80035b0 con 0x7fc2b8003f80 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2b8004850 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c370 tx=0x7fc2b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47337f18eeedd7dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0007c60 con 0x7fc2b8003f80 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0005960 con 0x7fc2b8003f80 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2cd59a700 1 -- 172.21.15.144:0/4240639137 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0005d10 con 0x7fc2b8003f80 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2d4c41700 1 -- 172.21.15.144:0/4240639137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 msgr2=0x7fc2b8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/4240639137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2b8004850 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c370 tx=0x7fc2b8005880 comp rx=0 tx=0).stop 2023-11-28T11:51:40.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.355+0000 7fc2ced9d700 1 -- 172.21.15.144:0/4240639137 reap_dead start 2023-11-28T11:51:40.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.356+0000 7fc2d4c41700 1 -- 172.21.15.144:0/4240639137 shutdown_connections 2023-11-28T11:51:40.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.356+0000 7fc2d4c41700 1 -- 172.21.15.144:0/4240639137 >> 172.21.15.144:0/4240639137 conn(0x7fc2d006aed0 msgr2=0x7fc2d006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:40.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.356+0000 7fc2d4c41700 1 -- 172.21.15.144:0/4240639137 shutdown_connections 2023-11-28T11:51:40.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.356+0000 7fc2d4c41700 1 -- 172.21.15.144:0/4240639137 wait complete. 2023-11-28T11:51:40.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.357+0000 7fc2d4c41700 1 Processor -- start 2023-11-28T11:51:40.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.357+0000 7fc2d4c41700 1 -- start start 2023-11-28T11:51:40.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2d4c41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2d00c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2d4c41700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2d00ca830 0x7fc2d01f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2d4c41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d01f2900 0x7fc2d00c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2d4c41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01f2470 con 0x7fc2b8003f80 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2d4c41700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01f2160 con 0x7fc2d00ca830 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2d4c41700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01f5c90 con 0x7fc2d01f2900 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2ce59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2d00c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2ce59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2d00c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35234/0 (socket says 172.21.15.144:35234) 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.358+0000 7fc2ce59c700 1 -- 172.21.15.144:0/196865881 learned_addr learned my addr 172.21.15.144:0/196865881 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2d00ca830 0x7fc2d01f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ced9d700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d01f2900 0x7fc2d00c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ce59c700 1 -- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d01f2900 msgr2=0x7fc2d00c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ce59c700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d01f2900 0x7fc2d00c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ce59c700 1 -- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2d00ca830 msgr2=0x7fc2d01f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ce59c700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2d00ca830 0x7fc2d01f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ce59c700 1 -- 172.21.15.144:0/196865881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d006bcc0 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2ce59c700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2d00c4900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4011a20 tx=0x7fc2c4011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4005c30 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c4005d90 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d006b9e0 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400f950 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.359+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d006bfa0 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.360+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d0068690 con 0x7fc2b8003f80 2023-11-28T11:51:40.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.360+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2c400fab0 con 0x7fc2b8003f80 2023-11-28T11:51:40.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.361+0000 7fc2beffd700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2b00682f0 0x7fc2b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.361+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2c4086400 con 0x7fc2b8003f80 2023-11-28T11:51:40.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.361+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2b00682f0 0x7fc2b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.361+0000 7fc2cdd9b700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2b00682f0 0x7fc2b006a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0015040 tx=0x7fc2c000a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.365+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2c4052d20 con 0x7fc2b8003f80 2023-11-28T11:51:40.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.548+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2d015d210 con 0x7fc2b8003f80 2023-11-28T11:51:40.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.549+0000 7fc2beffd700 1 -- 172.21.15.144:0/196865881 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2c4056340 con 0x7fc2b8003f80 2023-11-28T11:51:40.550 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:40.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2b00682f0 msgr2=0x7fc2b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2b00682f0 0x7fc2b006a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0015040 tx=0x7fc2c000a000 comp rx=0 tx=0).stop 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 msgr2=0x7fc2d00c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2d00c4900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4011a20 tx=0x7fc2c4011d30 comp rx=0 tx=0).stop 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 shutdown_connections 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2b8003f80 0x7fc2d00c4900 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2b00682f0 0x7fc2b006a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2d01f2900 0x7fc2d00c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 --2- 172.21.15.144:0/196865881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2d00ca830 0x7fc2d01f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 >> 172.21.15.144:0/196865881 conn(0x7fc2d006aed0 msgr2=0x7fc2d015ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 shutdown_connections 2023-11-28T11:51:40.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.551+0000 7fc2d4c41700 1 -- 172.21.15.144:0/196865881 wait complete. 2023-11-28T11:51:40.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:40.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.686+0000 7f5c7c768700 1 Processor -- start 2023-11-28T11:51:40.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.686+0000 7f5c7c768700 1 -- start start 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7c768700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c74071040 0x7f5c74071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7c768700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c741609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7c768700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c74071950 con 0x7f5c7406dee0 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7c768700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7415d060 con 0x7f5c74071040 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7c768700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7415d340 con 0x7f5c74160540 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c74071040 0x7f5c74071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c74071040 0x7f5c74071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35242/0 (socket says 172.21.15.144:35242) 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 -- 172.21.15.144:0/2558046531 learned_addr learned my addr 172.21.15.144:0/2558046531 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7ad05700 1 --2- 172.21.15.144:0/2558046531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c741609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 -- 172.21.15.144:0/2558046531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 msgr2=0x7f5c741609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 --2- 172.21.15.144:0/2558046531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c741609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 -- 172.21.15.144:0/2558046531 >> v1:172.21.15.177:6789/0 conn(0x7f5c7406dee0 legacy=0x7f5c7406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 -- 172.21.15.144:0/2558046531 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7415d620 con 0x7f5c74071040 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c7a504700 1 --2- 172.21.15.144:0/2558046531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c74071410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c640098f0 tx=0x7f5c64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a871d1def0cc9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c79502700 1 -- 172.21.15.144:0/2558046531 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004230 con 0x7f5c74071040 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c79502700 1 -- 172.21.15.144:0/2558046531 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c64004390 con 0x7f5c74071040 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.688+0000 7f5c79502700 1 -- 172.21.15.144:0/2558046531 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640036a0 con 0x7f5c74071040 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 -- 172.21.15.144:0/2558046531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 msgr2=0x7f5c74071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2558046531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c74071410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c640098f0 tx=0x7f5c64009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:40.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 -- 172.21.15.144:0/2558046531 shutdown_connections 2023-11-28T11:51:40.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2558046531 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c741609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2558046531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c74071410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 -- 172.21.15.144:0/2558046531 >> 172.21.15.144:0/2558046531 conn(0x7f5c740d0000 msgr2=0x7f5c740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:40.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 -- 172.21.15.144:0/2558046531 shutdown_connections 2023-11-28T11:51:40.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.689+0000 7f5c7c768700 1 -- 172.21.15.144:0/2558046531 wait complete. 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.690+0000 7f5c7c768700 1 Processor -- start 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.690+0000 7f5c7c768700 1 -- start start 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7c768700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7406dee0 0x7f5c740c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7c768700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c740c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7c768700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c740c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7c768700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74158dd0 con 0x7f5c74071040 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7c768700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740c4480 con 0x7f5c7406dee0 2023-11-28T11:51:40.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7c768700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74159020 con 0x7f5c74160540 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c740c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c740c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35248/0 (socket says 172.21.15.144:35248) 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 -- 172.21.15.144:0/2627920335 learned_addr learned my addr 172.21.15.144:0/2627920335 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7a504700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7406dee0 0x7f5c740c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7ad05700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c740c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 -- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 msgr2=0x7f5c740c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c740c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 -- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7406dee0 msgr2=0x7f5c740c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7406dee0 0x7f5c740c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c79d03700 1 -- 172.21.15.144:0/2627920335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64009550 con 0x7f5c74071040 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.691+0000 7f5c7ad05700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c740c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:40.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.692+0000 7f5c79d03700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c740c48d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b4a0 tx=0x7f5c7000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.692+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011830 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.692+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70011990 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.692+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7415ccd0 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.692+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70011d40 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.692+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7415d900 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.693+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7406a270 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.693+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c7000f050 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.693+0000 7f5c6affd700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c580682f0 0x7f5c5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.693+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c70085760 con 0x7f5c74071040 2023-11-28T11:51:40.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.693+0000 7f5c7a504700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c580682f0 0x7f5c5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:40.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.694+0000 7f5c7a504700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c580682f0 0x7f5c5806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009840 tx=0x7f5c64008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:40.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.698+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c70051fc0 con 0x7f5c74071040 2023-11-28T11:51:40.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.881+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c741594e0 con 0x7f5c74071040 2023-11-28T11:51:40.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.881+0000 7f5c6affd700 1 -- 172.21.15.144:0/2627920335 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c700555e0 con 0x7f5c74071040 2023-11-28T11:51:40.883 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:40.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c580682f0 msgr2=0x7f5c5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c580682f0 0x7f5c5806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009840 tx=0x7f5c64008030 comp rx=0 tx=0).stop 2023-11-28T11:51:40.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 msgr2=0x7f5c740c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:40.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c740c48d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b4a0 tx=0x7f5c7000b7b0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 shutdown_connections 2023-11-28T11:51:40.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c580682f0 0x7f5c5806a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c74160540 0x7f5c740c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7406dee0 0x7f5c740c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 --2- 172.21.15.144:0/2627920335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c74071040 0x7f5c740c48d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:40.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 >> 172.21.15.144:0/2627920335 conn(0x7f5c740d0000 msgr2=0x7f5c7415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:40.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 shutdown_connections 2023-11-28T11:51:40.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:40.883+0000 7f5c7c768700 1 -- 172.21.15.144:0/2627920335 wait complete. 2023-11-28T11:51:40.898 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:41.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.018+0000 7fcb57aec700 1 Processor -- start 2023-11-28T11:51:41.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.018+0000 7fcb57aec700 1 -- start start 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb57aec700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcb5006c990 0x7fcb5006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb57aec700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5006ae60 con 0x7fcb500ca240 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb57aec700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5006a270 con 0x7fcb5006e9f0 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb57aec700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5006d350 con 0x7fcb5006c990 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb56089700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcb500ca240 0x7fcb500ca630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51970/0 (socket says 172.21.15.144:51970) 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb56089700 1 -- 172.21.15.144:0/3685582251 learned_addr learned my addr 172.21.15.144:0/3685582251 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb55087700 1 --2- 172.21.15.144:0/3685582251 >> v2:172.21.15.177:3300/0 conn(0x7fcb5006c990 0x7fcb5006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978461590 0 0) 0x7fcb5006ae60 con 0x7fcb500ca240 2023-11-28T11:51:41.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.019+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb3c0035b0 con 0x7fcb500ca240 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 885896985 0 0) 0x7fcb3c0035b0 con 0x7fcb500ca240 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 >> v2:172.21.15.177:3300/0 conn(0x7fcb5006c990 msgr2=0x7fcb5006ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 --2- 172.21.15.144:0/3685582251 >> v2:172.21.15.177:3300/0 conn(0x7fcb5006c990 0x7fcb5006ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 >> v1:172.21.15.177:6789/0 conn(0x7fcb5006e9f0 legacy=0x7fcb5006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5006a530 con 0x7fcb500ca240 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fcb480019f0 con 0x7fcb500ca240 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 >> v1:172.21.15.144:6789/0 conn(0x7fcb500ca240 legacy=0x7fcb500ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb3c003a00 0x7fcb3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb3c004850 0x7fcb3c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb3c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5006d350 con 0x7fcb3c005200 2023-11-28T11:51:41.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c00c080 con 0x7fcb3c003a00 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c00c240 con 0x7fcb3c004850 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb55087700 1 --2- 172.21.15.144:0/3685582251 >> v2:172.21.15.177:3300/0 conn(0x7fcb5006c990 0x7fcb5006ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb3c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb55888700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb3c003a00 0x7fcb3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb55087700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb3c004850 0x7fcb3c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 -- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb3c004850 msgr2=0x7fcb3c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb3c004850 0x7fcb3c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 -- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb3c003a00 msgr2=0x7fcb3c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb3c003a00 0x7fcb3c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 -- 172.21.15.144:0/3685582251 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0035b0 con 0x7fcb3c005200 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 -- 172.21.15.144:0/3685582251 reap_dead start 2023-11-28T11:51:41.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb56089700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb3c00ba10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800a7f0 tx=0x7fcb3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dfd98ebd0e4ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800a2f0 con 0x7fcb3c005200 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48012620 con 0x7fcb3c005200 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.020+0000 7fcb54886700 1 -- 172.21.15.144:0/3685582251 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480129d0 con 0x7fcb3c005200 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 -- 172.21.15.144:0/3685582251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 msgr2=0x7fcb3c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb3c00ba10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800a7f0 tx=0x7fcb3c005880 comp rx=0 tx=0).stop 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 -- 172.21.15.144:0/3685582251 shutdown_connections 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 --2- 172.21.15.144:0/3685582251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb3c00ba10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 -- 172.21.15.144:0/3685582251 >> 172.21.15.144:0/3685582251 conn(0x7fcb500b12b0 msgr2=0x7fcb500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 -- 172.21.15.144:0/3685582251 shutdown_connections 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.021+0000 7fcb57aec700 1 -- 172.21.15.144:0/3685582251 wait complete. 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.022+0000 7fcb57aec700 1 Processor -- start 2023-11-28T11:51:41.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.022+0000 7fcb57aec700 1 -- start start 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb57aec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb500c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb57aec700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb500b41f0 0x7fcb501503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb57aec700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb500af2e0 0x7fcb500c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb57aec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5015b880 con 0x7fcb3c005200 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb57aec700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5015bb60 con 0x7fcb500af2e0 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb57aec700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5015be40 con 0x7fcb500b41f0 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb500c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb500c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35266/0 (socket says 172.21.15.144:35266) 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 -- 172.21.15.144:0/1523680222 learned_addr learned my addr 172.21.15.144:0/1523680222 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55087700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb500b41f0 0x7fcb501503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb56089700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb500af2e0 0x7fcb500c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 -- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb500b41f0 msgr2=0x7fcb501503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb500b41f0 0x7fcb501503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 -- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb500af2e0 msgr2=0x7fcb500c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb500af2e0 0x7fcb500c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 -- 172.21.15.144:0/1523680222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5006aaf0 con 0x7fcb3c005200 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb55888700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb500c6030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000f820 tx=0x7fcb4000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.023+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400052e0 con 0x7fcb3c005200 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.024+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb40005440 con 0x7fcb3c005200 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.024+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5006a810 con 0x7fcb3c005200 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.024+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400038a0 con 0x7fcb3c005200 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.024+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5006edc0 con 0x7fcb3c005200 2023-11-28T11:51:41.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.024+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb500cd5b0 con 0x7fcb3c005200 2023-11-28T11:51:41.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.025+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb40003a00 con 0x7fcb3c005200 2023-11-28T11:51:41.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.025+0000 7fcb467fc700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcb340682f0 0x7fcb3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.025+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb40085a20 con 0x7fcb3c005200 2023-11-28T11:51:41.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.025+0000 7fcb55087700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcb340682f0 0x7fcb3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.026+0000 7fcb55087700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcb340682f0 0x7fcb3406a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009470 tx=0x7fcb4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.029+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb40052330 con 0x7fcb3c005200 2023-11-28T11:51:41.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.211+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb500cb1d0 con 0x7fcb3c005200 2023-11-28T11:51:41.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.212+0000 7fcb467fc700 1 -- 172.21.15.144:0/1523680222 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fcb40055950 con 0x7fcb3c005200 2023-11-28T11:51:41.213 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:41.213 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:41.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcb340682f0 msgr2=0x7fcb3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcb340682f0 0x7fcb3406a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009470 tx=0x7fcb4c008040 comp rx=0 tx=0).stop 2023-11-28T11:51:41.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 msgr2=0x7fcb500c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb500c6030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000f820 tx=0x7fcb4000fb30 comp rx=0 tx=0).stop 2023-11-28T11:51:41.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 shutdown_connections 2023-11-28T11:51:41.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb3c005200 0x7fcb500c6030 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcb340682f0 0x7fcb3406a7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb500af2e0 0x7fcb500c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 --2- 172.21.15.144:0/1523680222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb500b41f0 0x7fcb501503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 >> 172.21.15.144:0/1523680222 conn(0x7fcb500b12b0 msgr2=0x7fcb500b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 shutdown_connections 2023-11-28T11:51:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.215+0000 7fcb57aec700 1 -- 172.21.15.144:0/1523680222 wait complete. 2023-11-28T11:51:41.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:41.229 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:41.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.351+0000 7fe43f6da700 1 Processor -- start 2023-11-28T11:51:41.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.352+0000 7fe43f6da700 1 -- start start 2023-11-28T11:51:41.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.352+0000 7fe43f6da700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe438071040 0x7fe438071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.352+0000 7fe43f6da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 0x7fe4381609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.352+0000 7fe43f6da700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe438071950 con 0x7fe43806dee0 2023-11-28T11:51:41.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.352+0000 7fe43f6da700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43815d060 con 0x7fe438071040 2023-11-28T11:51:41.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.352+0000 7fe43f6da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43815d340 con 0x7fe438160540 2023-11-28T11:51:41.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43cc75700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe43806dee0 0x7fe43806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:51986/0 (socket says 172.21.15.144:51986) 2023-11-28T11:51:41.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43cc75700 1 -- 172.21.15.144:0/578993580 learned_addr learned my addr 172.21.15.144:0/578993580 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:41.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247678429 0 0) 0x7fe438071950 con 0x7fe43806dee0 2023-11-28T11:51:41.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43d476700 1 --2- 172.21.15.144:0/578993580 >> v2:172.21.15.177:3300/0 conn(0x7fe438071040 0x7fe438071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4240035b0 con 0x7fe43806dee0 2023-11-28T11:51:41.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43dc77700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 0x7fe4381609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827758604 0 0) 0x7fe4240035b0 con 0x7fe43806dee0 2023-11-28T11:51:41.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 msgr2=0x7fe4381609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 0x7fe4381609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 >> v2:172.21.15.177:3300/0 conn(0x7fe438071040 msgr2=0x7fe438071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 --2- 172.21.15.144:0/578993580 >> v2:172.21.15.177:3300/0 conn(0x7fe438071040 0x7fe438071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43815d620 con 0x7fe43806dee0 2023-11-28T11:51:41.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fe434001670 con 0x7fe43806dee0 2023-11-28T11:51:41.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 >> v1:172.21.15.144:6789/0 conn(0x7fe43806dee0 legacy=0x7fe43806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424003a00 0x7fe424003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe424004850 0x7fe424004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 0x7fe424005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43815d060 con 0x7fe424004850 2023-11-28T11:51:41.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43815d340 con 0x7fe424003a00 2023-11-28T11:51:41.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42400ba90 con 0x7fe438160540 2023-11-28T11:51:41.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43cc75700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe424004850 0x7fe424004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43d476700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424003a00 0x7fe424003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.353+0000 7fe43dc77700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 0x7fe424005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43cc75700 1 -- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 msgr2=0x7fe424005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43cc75700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe438160540 0x7fe424005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43cc75700 1 -- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424003a00 msgr2=0x7fe424003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43cc75700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424003a00 0x7fe424003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43cc75700 1 -- 172.21.15.144:0/578993580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240035b0 con 0x7fe424004850 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43d476700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424003a00 0x7fe424003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43cc75700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe424004850 0x7fe424004cc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c320 tx=0x7fe424005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cde24fc65318b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43400ce60 con 0x7fe424004850 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434005960 con 0x7fe424004850 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe42ffff700 1 -- 172.21.15.144:0/578993580 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434005cf0 con 0x7fe424004850 2023-11-28T11:51:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43f6da700 1 -- 172.21.15.144:0/578993580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe424004850 msgr2=0x7fe424004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43f6da700 1 --2- 172.21.15.144:0/578993580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe424004850 0x7fe424004cc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c320 tx=0x7fe424005880 comp rx=0 tx=0).stop 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43dc77700 1 -- 172.21.15.144:0/578993580 reap_dead start 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43f6da700 1 -- 172.21.15.144:0/578993580 shutdown_connections 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.354+0000 7fe43f6da700 1 -- 172.21.15.144:0/578993580 >> 172.21.15.144:0/578993580 conn(0x7fe4380d0000 msgr2=0x7fe4380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.355+0000 7fe43f6da700 1 -- 172.21.15.144:0/578993580 shutdown_connections 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.355+0000 7fe43f6da700 1 -- 172.21.15.144:0/578993580 wait complete. 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.356+0000 7fe43f6da700 1 Processor -- start 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.356+0000 7fe43f6da700 1 -- start start 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.356+0000 7fe43f6da700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424004850 0x7fe4381530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4380c3a60 0x7fe4380c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 0x7fe4380c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4381564e0 con 0x7fe4380c3ed0 2023-11-28T11:51:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe438156aa0 con 0x7fe424004850 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe438153630 con 0x7fe4380c3a60 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 0x7fe4380c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 0x7fe4380c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35288/0 (socket says 172.21.15.144:35288) 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 -- 172.21.15.144:0/3836971610 learned_addr learned my addr 172.21.15.144:0/3836971610 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43cc75700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4380c3a60 0x7fe4380c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43d476700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424004850 0x7fe4381530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 -- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4380c3a60 msgr2=0x7fe4380c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4380c3a60 0x7fe4380c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 -- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424004850 msgr2=0x7fe4381530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424004850 0x7fe4381530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 -- 172.21.15.144:0/3836971610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43815ccd0 con 0x7fe4380c3ed0 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43dc77700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 0x7fe4380c4ce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe438071d40 tx=0x7fe430011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43000e040 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43000f040 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43815d900 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43000f940 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.357+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43806a270 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.358+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe438156c30 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.359+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe430025070 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.359+0000 7fe42dffb700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe41c0682f0 0x7fe41c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.359+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe430085f70 con 0x7fe4380c3ed0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.359+0000 7fe43d476700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe41c0682f0 0x7fe41c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.360+0000 7fe43d476700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe41c0682f0 0x7fe41c06a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4380c6a40 tx=0x7fe428009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.363+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4300b75e0 con 0x7fe4380c3ed0 2023-11-28T11:51:41.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.544+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe438155af0 con 0x7fe4380c3ed0 2023-11-28T11:51:41.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.544+0000 7fe42dffb700 1 -- 172.21.15.144:0/3836971610 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe4300527d0 con 0x7fe4380c3ed0 2023-11-28T11:51:41.545 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe41c0682f0 msgr2=0x7fe41c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe41c0682f0 0x7fe41c06a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe4380c6a40 tx=0x7fe428009210 comp rx=0 tx=0).stop 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 msgr2=0x7fe4380c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 0x7fe4380c4ce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe438071d40 tx=0x7fe430011f60 comp rx=0 tx=0).stop 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 shutdown_connections 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4380c3ed0 0x7fe4380c4ce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe41c0682f0 0x7fe41c06a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe424004850 0x7fe4381530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 --2- 172.21.15.144:0/3836971610 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4380c3a60 0x7fe4380c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 >> 172.21.15.144:0/3836971610 conn(0x7fe4380d0000 msgr2=0x7fe43806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:41.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 shutdown_connections 2023-11-28T11:51:41.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.547+0000 7fe43f6da700 1 -- 172.21.15.144:0/3836971610 wait complete. 2023-11-28T11:51:41.560 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:41.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.682+0000 7fc99531c700 1 Processor -- start 2023-11-28T11:51:41.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.682+0000 7fc99531c700 1 -- start start 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc99531c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9900cd7c0 0x7fc9900cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc99531c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9900d1950 con 0x7fc99015c9b0 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc99531c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9900ce140 con 0x7fc990161340 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc99531c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9900b5800 con 0x7fc9900cd7c0 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9900cd7c0 0x7fc9900cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc9900cd7c0 0x7fc9900cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35296/0 (socket says 172.21.15.144:35296) 2023-11-28T11:51:41.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 -- 172.21.15.144:0/2879519361 learned_addr learned my addr 172.21.15.144:0/2879519361 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:41.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 -- 172.21.15.144:0/2879519361 >> v1:172.21.15.177:6789/0 conn(0x7fc990161340 legacy=0x7fc9900d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 -- 172.21.15.144:0/2879519361 >> v1:172.21.15.144:6789/0 conn(0x7fc99015c9b0 legacy=0x7fc9900b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 -- 172.21.15.144:0/2879519361 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900d24f0 con 0x7fc9900cd7c0 2023-11-28T11:51:41.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98f7fe700 1 --2- 172.21.15.144:0/2879519361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9900cd7c0 0x7fc9900cdc00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc988009e50 tx=0x7fc98800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280617182f94a14a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.683+0000 7fc98dffb700 1 -- 172.21.15.144:0/2879519361 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800e070 con 0x7fc9900cd7c0 2023-11-28T11:51:41.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc98dffb700 1 -- 172.21.15.144:0/2879519361 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800b9c0 con 0x7fc9900cd7c0 2023-11-28T11:51:41.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc98dffb700 1 -- 172.21.15.144:0/2879519361 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800bd70 con 0x7fc9900cd7c0 2023-11-28T11:51:41.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 -- 172.21.15.144:0/2879519361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9900cd7c0 msgr2=0x7fc9900cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 --2- 172.21.15.144:0/2879519361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9900cd7c0 0x7fc9900cdc00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc988009e50 tx=0x7fc98800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:41.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 -- 172.21.15.144:0/2879519361 shutdown_connections 2023-11-28T11:51:41.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 --2- 172.21.15.144:0/2879519361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9900cd7c0 0x7fc9900cdc00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 -- 172.21.15.144:0/2879519361 >> 172.21.15.144:0/2879519361 conn(0x7fc990072600 msgr2=0x7fc9900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:41.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 -- 172.21.15.144:0/2879519361 shutdown_connections 2023-11-28T11:51:41.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.684+0000 7fc99531c700 1 -- 172.21.15.144:0/2879519361 wait complete. 2023-11-28T11:51:41.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.685+0000 7fc99531c700 1 Processor -- start 2023-11-28T11:51:41.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.685+0000 7fc99531c700 1 -- start start 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc99531c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9900cd7c0 0x7fc990159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc99531c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 0x7fc9900cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc99531c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc990161340 0x7fc9900ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc99531c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900cd170 con 0x7fc99015c9b0 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc99531c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901e66f0 con 0x7fc990161340 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc99531c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901e69d0 con 0x7fc9900cd7c0 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc98e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 0x7fc9900cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc98e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 0x7fc9900cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35300/0 (socket says 172.21.15.144:35300) 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc98e7fc700 1 -- 172.21.15.144:0/2675773100 learned_addr learned my addr 172.21.15.144:0/2675773100 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:41.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc98f7fe700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc990161340 0x7fc9900ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.686+0000 7fc98effd700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9900cd7c0 0x7fc990159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc98e7fc700 1 -- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9900cd7c0 msgr2=0x7fc990159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc98e7fc700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9900cd7c0 0x7fc990159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc98e7fc700 1 -- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc990161340 msgr2=0x7fc9900ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc98e7fc700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc990161340 0x7fc9900ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc98e7fc700 1 -- 172.21.15.144:0/2675773100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988009b30 con 0x7fc99015c9b0 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc98e7fc700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 0x7fc9900cc6f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c4e0 tx=0x7fc98000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800052d0 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980005430 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9900d1300 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800038c0 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.687+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9900b5520 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.688+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9900d15e0 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.688+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc980019030 con 0x7fc99015c9b0 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.689+0000 7fc9777fe700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc97c0682f0 0x7fc97c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:41.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.689+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc980084ef0 con 0x7fc99015c9b0 2023-11-28T11:51:41.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.689+0000 7fc98effd700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc97c0682f0 0x7fc97c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:41.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.689+0000 7fc98effd700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc97c0682f0 0x7fc97c06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc984007fd0 tx=0x7fc984009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:41.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.693+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc980051750 con 0x7fc99015c9b0 2023-11-28T11:51:41.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.876+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9900745c0 con 0x7fc99015c9b0 2023-11-28T11:51:41.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.877+0000 7fc9777fe700 1 -- 172.21.15.144:0/2675773100 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9800170a0 con 0x7fc99015c9b0 2023-11-28T11:51:41.878 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:41.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc97c0682f0 msgr2=0x7fc97c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc97c0682f0 0x7fc97c06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc984007fd0 tx=0x7fc984009040 comp rx=0 tx=0).stop 2023-11-28T11:51:41.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 msgr2=0x7fc9900cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:41.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 0x7fc9900cc6f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c4e0 tx=0x7fc98000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 shutdown_connections 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc97c0682f0 0x7fc97c06a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc990161340 0x7fc9900ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9900cd7c0 0x7fc990159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 --2- 172.21.15.144:0/2675773100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc99015c9b0 0x7fc9900cc6f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.878+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 >> 172.21.15.144:0/2675773100 conn(0x7fc990072600 msgr2=0x7fc9900b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.879+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 shutdown_connections 2023-11-28T11:51:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:41.879+0000 7fc99531c700 1 -- 172.21.15.144:0/2675773100 wait complete. 2023-11-28T11:51:41.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.014+0000 7f7d03f08700 1 Processor -- start 2023-11-28T11:51:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.014+0000 7f7d03f08700 1 -- start start 2023-11-28T11:51:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d03f08700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc140950 0x7f7cfc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d03f08700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cfc0b2e70 0x7f7cfc06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d03f08700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc13ff00 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d03f08700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc1404c0 con 0x7f7cfc0b2e70 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d03f08700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc140dc0 con 0x7f7cfc140950 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d024a5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cfc0b2e70 0x7f7cfc06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d01ca4700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7cfc0cd8c0 0x7f7cfc0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52018/0 (socket says 172.21.15.144:52018) 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d024a5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cfc0b2e70 0x7f7cfc06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35310/0 (socket says 172.21.15.144:35310) 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d01ca4700 1 -- 172.21.15.144:0/2296793980 learned_addr learned my addr 172.21.15.144:0/2296793980 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d014a3700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc140950 0x7f7cfc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d024a5700 1 -- 172.21.15.144:0/2296793980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc140950 msgr2=0x7f7cfc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d024a5700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc140950 0x7f7cfc0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d024a5700 1 -- 172.21.15.144:0/2296793980 >> v1:172.21.15.144:6789/0 conn(0x7f7cfc0cd8c0 legacy=0x7f7cfc0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d024a5700 1 -- 172.21.15.144:0/2296793980 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc15aa80 con 0x7f7cfc0b2e70 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.015+0000 7f7d014a3700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc140950 0x7f7cfc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d024a5700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0b2e70 0x7f7cfc06f850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009d90 tx=0x7f7cf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef81bad81e20523f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d00ca2700 1 -- 172.21.15.144:0/2296793980 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400e070 con 0x7f7cfc0b2e70 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d00ca2700 1 -- 172.21.15.144:0/2296793980 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf400bbd0 con 0x7f7cfc0b2e70 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d00ca2700 1 -- 172.21.15.144:0/2296793980 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4012430 con 0x7f7cfc0b2e70 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 -- 172.21.15.144:0/2296793980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0b2e70 msgr2=0x7f7cfc06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0b2e70 0x7f7cfc06f850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009d90 tx=0x7f7cf400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 -- 172.21.15.144:0/2296793980 shutdown_connections 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0b2e70 0x7f7cfc06f850 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 --2- 172.21.15.144:0/2296793980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc140950 0x7f7cfc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 -- 172.21.15.144:0/2296793980 >> 172.21.15.144:0/2296793980 conn(0x7f7cfc06aed0 msgr2=0x7f7cfc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:42.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 -- 172.21.15.144:0/2296793980 shutdown_connections 2023-11-28T11:51:42.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.016+0000 7f7d03f08700 1 -- 172.21.15.144:0/2296793980 wait complete. 2023-11-28T11:51:42.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 Processor -- start 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 -- start start 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc0b2e70 0x7f7cfc159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 0x7f7cfc15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cfc140950 0x7f7cfc1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15c1e0 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15b370 con 0x7f7cfc140950 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d03f08700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15b650 con 0x7f7cfc0b2e70 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d014a3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 0x7f7cfc15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d014a3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 0x7f7cfc15ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35314/0 (socket says 172.21.15.144:35314) 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.018+0000 7f7d014a3700 1 -- 172.21.15.144:0/67000061 learned_addr learned my addr 172.21.15.144:0/67000061 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d01ca4700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc0b2e70 0x7f7cfc159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d024a5700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cfc140950 0x7f7cfc1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d014a3700 1 -- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc0b2e70 msgr2=0x7f7cfc159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d014a3700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc0b2e70 0x7f7cfc159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d014a3700 1 -- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cfc140950 msgr2=0x7f7cfc1f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d014a3700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cfc140950 0x7f7cfc1f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d014a3700 1 -- 172.21.15.144:0/67000061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009a70 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d024a5700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cfc140950 0x7f7cfc1f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d014a3700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 0x7f7cfc15ac70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c7f0 tx=0x7f7cf800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8011800 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8011960 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0b35f0 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8011d10 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.019+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc0b3310 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.020+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc140150 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.020+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7cf8017030 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.021+0000 7f7cf27fc700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ce00682f0 0x7f7ce006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.021+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7cf8085760 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.021+0000 7f7d01ca4700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ce00682f0 0x7f7ce006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.021+0000 7f7d01ca4700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ce00682f0 0x7f7ce006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007fd0 tx=0x7f7cec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.025+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7cf8051f40 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.205+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7cfc068690 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.205+0000 7f7cf27fc700 1 -- 172.21.15.144:0/67000061 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7cf8055560 con 0x7f7cfc0cd8c0 2023-11-28T11:51:42.206 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ce00682f0 msgr2=0x7f7ce006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ce00682f0 0x7f7ce006a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7cec007fd0 tx=0x7f7cec009040 comp rx=0 tx=0).stop 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 msgr2=0x7f7cfc15ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 0x7f7cfc15ac70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800c7f0 tx=0x7f7cf800cb00 comp rx=0 tx=0).stop 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 shutdown_connections 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ce00682f0 0x7f7ce006a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cfc140950 0x7f7cfc1f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.207+0000 7f7d03f08700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cfc0b2e70 0x7f7cfc159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.208+0000 7f7d03f08700 1 --2- 172.21.15.144:0/67000061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc0cd8c0 0x7f7cfc15ac70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.208+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 >> 172.21.15.144:0/67000061 conn(0x7f7cfc06aed0 msgr2=0x7f7cfc0cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.208+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 shutdown_connections 2023-11-28T11:51:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.208+0000 7f7d03f08700 1 -- 172.21.15.144:0/67000061 wait complete. 2023-11-28T11:51:42.224 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:42.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.345+0000 7f1af8d2a700 1 Processor -- start 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.345+0000 7f1af8d2a700 1 -- start start 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.346+0000 7f1af8d2a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.346+0000 7f1af8d2a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1af406dee0 0x7f1af406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.346+0000 7f1af8d2a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1af406cdc0 0x7f1af406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.346+0000 7f1af8d2a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af4071950 con 0x7f1af406dee0 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.346+0000 7f1af8d2a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af415d060 con 0x7f1af406cdc0 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.346+0000 7f1af8d2a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af415d340 con 0x7f1af4071040 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1af406dee0 0x7f1af406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1af406dee0 0x7f1af406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35320/0 (socket says 172.21.15.144:35320) 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 -- 172.21.15.144:0/1061842517 learned_addr learned my addr 172.21.15.144:0/1061842517 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:42.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af2d9d700 1 --2- 172.21.15.144:0/1061842517 >> v2:172.21.15.177:3300/0 conn(0x7f1af406cdc0 0x7f1af406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af259c700 1 --2- 172.21.15.144:0/1061842517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 -- 172.21.15.144:0/1061842517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 msgr2=0x7f1af4071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:42.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1061842517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 -- 172.21.15.144:0/1061842517 >> v2:172.21.15.177:3300/0 conn(0x7f1af406cdc0 msgr2=0x7f1af406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:42.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1061842517 >> v2:172.21.15.177:3300/0 conn(0x7f1af406cdc0 0x7f1af406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 -- 172.21.15.144:0/1061842517 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af415d620 con 0x7f1af406dee0 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1061842517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406dee0 0x7f1af406e350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1af4071d40 tx=0x7f1ae4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eabb915dd71cb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af159a700 1 -- 172.21.15.144:0/1061842517 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4011830 con 0x7f1af406dee0 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af159a700 1 -- 172.21.15.144:0/1061842517 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae4011990 con 0x7f1af406dee0 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.347+0000 7f1af159a700 1 -- 172.21.15.144:0/1061842517 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4011d20 con 0x7f1af406dee0 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1061842517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406dee0 msgr2=0x7f1af406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1061842517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406dee0 0x7f1af406e350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1af4071d40 tx=0x7f1ae4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1061842517 shutdown_connections 2023-11-28T11:51:42.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1061842517 >> v2:172.21.15.177:3300/0 conn(0x7f1af406cdc0 0x7f1af406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1061842517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1061842517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406dee0 0x7f1af406e350 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1061842517 >> 172.21.15.144:0/1061842517 conn(0x7f1af40d0000 msgr2=0x7f1af40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:42.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1061842517 shutdown_connections 2023-11-28T11:51:42.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.348+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1061842517 wait complete. 2023-11-28T11:51:42.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.349+0000 7f1af8d2a700 1 Processor -- start 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.349+0000 7f1af8d2a700 1 -- start start 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af8d2a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 0x7f1af40b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af8d2a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af406dee0 0x7f1af40c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af8d2a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af8d2a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40c4b20 con 0x7f1af406cdc0 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af8d2a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40ad500 con 0x7f1af406dee0 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af8d2a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40acf10 con 0x7f1af4071040 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 0x7f1af40b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 0x7f1af40b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35326/0 (socket says 172.21.15.144:35326) 2023-11-28T11:51:42.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 -- 172.21.15.144:0/1287836371 learned_addr learned my addr 172.21.15.144:0/1287836371 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af406dee0 0x7f1af40c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af2d9d700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 -- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 msgr2=0x7f1af4157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 -- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af406dee0 msgr2=0x7f1af40c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af406dee0 0x7f1af40c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af259c700 1 -- 172.21.15.144:0/1287836371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4009550 con 0x7f1af406cdc0 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af406dee0 0x7f1af40c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.350+0000 7f1af2d9d700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:42.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.351+0000 7f1af259c700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 0x7f1af40b2e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800e8a0 tx=0x7f1ae800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.351+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8009590 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.351+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae800eea0 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.351+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af406e890 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.351+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800ca50 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.351+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af415d900 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.352+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af415ccd0 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.352+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ae8019030 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.353+0000 7f1ae2ffd700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1ad40682f0 0x7f1ad406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.353+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ae8085be0 con 0x7f1af406cdc0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.353+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1ad40682f0 0x7f1ad406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.353+0000 7f1af1d9b700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1ad40682f0 0x7f1ad406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4000c00 tx=0x7f1ae4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.357+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ae8052350 con 0x7f1af406cdc0 2023-11-28T11:51:42.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.539+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1af406b280 con 0x7f1af406cdc0 2023-11-28T11:51:42.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.540+0000 7f1ae2ffd700 1 -- 172.21.15.144:0/1287836371 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ae8055970 con 0x7f1af406cdc0 2023-11-28T11:51:42.541 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.541+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1ad40682f0 msgr2=0x7f1ad406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1ad40682f0 0x7f1ad406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4000c00 tx=0x7f1ae4008030 comp rx=0 tx=0).stop 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 msgr2=0x7f1af40b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 0x7f1af40b2e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800e8a0 tx=0x7f1ae800ec60 comp rx=0 tx=0).stop 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 shutdown_connections 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af406cdc0 0x7f1af40b2e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1ad40682f0 0x7f1ad406a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1af4071040 0x7f1af4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 --2- 172.21.15.144:0/1287836371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af406dee0 0x7f1af40c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 >> 172.21.15.144:0/1287836371 conn(0x7f1af40d0000 msgr2=0x7f1af415f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:42.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 shutdown_connections 2023-11-28T11:51:42.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.542+0000 7f1af8d2a700 1 -- 172.21.15.144:0/1287836371 wait complete. 2023-11-28T11:51:42.555 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:42.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.674+0000 7f6e1b9f7700 1 Processor -- start 2023-11-28T11:51:42.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.674+0000 7f6e1b9f7700 1 -- start start 2023-11-28T11:51:42.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e1b9f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 0x7f6e140b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e1b9f7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1413ff00 con 0x7f6e140cd8c0 2023-11-28T11:51:42.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e1b9f7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e141404c0 con 0x7f6e140b2e70 2023-11-28T11:51:42.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e1b9f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14140dc0 con 0x7f6e14140950 2023-11-28T11:51:42.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e19793700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6e140cd8c0 0x7f6e140cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52046/0 (socket says 172.21.15.144:52046) 2023-11-28T11:51:42.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e19793700 1 -- 172.21.15.144:0/3479570652 learned_addr learned my addr 172.21.15.144:0/3479570652 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:42.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287917647 0 0) 0x7f6e1413ff00 con 0x7f6e140cd8c0 2023-11-28T11:51:42.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 0x7f6e140b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e000035b0 con 0x7f6e140cd8c0 2023-11-28T11:51:42.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071153562 0 0) 0x7f6e000035b0 con 0x7f6e140cd8c0 2023-11-28T11:51:42.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 msgr2=0x7f6e140b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 0x7f6e140b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 >> v1:172.21.15.177:6789/0 conn(0x7f6e140b2e70 legacy=0x7f6e1406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1415aa80 con 0x7f6e140cd8c0 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f6e100017f0 con 0x7f6e140cd8c0 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 >> v1:172.21.15.144:6789/0 conn(0x7f6e140cd8c0 legacy=0x7f6e140cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00003a30 0x7f6e00003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e00004850 0x7f6e00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 0x7f6e00005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14140dc0 con 0x7f6e00004850 2023-11-28T11:51:42.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0000bb10 con 0x7f6e00003a30 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0000bcd0 con 0x7f6e14140950 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.675+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e00004850 0x7f6e00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e19793700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00003a30 0x7f6e00003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e19f94700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 0x7f6e00005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e18f92700 1 -- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 msgr2=0x7f6e00005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e14140950 0x7f6e00005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e18f92700 1 -- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00003a30 msgr2=0x7f6e00003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00003a30 0x7f6e00003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e18f92700 1 -- 172.21.15.144:0/3479570652 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e000035b0 con 0x7f6e00004850 2023-11-28T11:51:42.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e00004850 0x7f6e00004cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009b30 tx=0x7f6e00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a106716757e72f22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400e6c0 con 0x7f6e00004850 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0400e820 con 0x7f6e00004850 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e0bfff700 1 -- 172.21.15.144:0/3479570652 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400eb90 con 0x7f6e00004850 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3479570652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e00004850 msgr2=0x7f6e00004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3479570652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e00004850 0x7f6e00004cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009b30 tx=0x7f6e00005880 comp rx=0 tx=0).stop 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e19f94700 1 -- 172.21.15.144:0/3479570652 reap_dead start 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3479570652 shutdown_connections 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.676+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3479570652 >> 172.21.15.144:0/3479570652 conn(0x7f6e1406aed0 msgr2=0x7f6e1406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.677+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3479570652 shutdown_connections 2023-11-28T11:51:42.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.677+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3479570652 wait complete. 2023-11-28T11:51:42.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.678+0000 7f6e1b9f7700 1 Processor -- start 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.678+0000 7f6e1b9f7700 1 -- start start 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00004850 0x7f6e14156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 0x7f6e1413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e1413d230 0x7f6e1406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1413d6a0 con 0x7f6e14156da0 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141432a0 con 0x7f6e00004850 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14143580 con 0x7f6e1413d230 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 0x7f6e1413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 0x7f6e1413ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35348/0 (socket says 172.21.15.144:35348) 2023-11-28T11:51:42.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 -- 172.21.15.144:0/3398222255 learned_addr learned my addr 172.21.15.144:0/3398222255 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e19793700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00004850 0x7f6e14156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e19f94700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e1413d230 0x7f6e1406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 -- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e1413d230 msgr2=0x7f6e1406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e1413d230 0x7f6e1406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 -- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00004850 msgr2=0x7f6e14156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00004850 0x7f6e14156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 -- 172.21.15.144:0/3398222255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140b35f0 con 0x7f6e14156da0 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e18f92700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 0x7f6e1413ccf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e04000c00 tx=0x7f6e040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400e6c0 con 0x7f6e14156da0 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04013d80 con 0x7f6e14156da0 2023-11-28T11:51:42.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e140b3310 con 0x7f6e14156da0 2023-11-28T11:51:42.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.679+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04028690 con 0x7f6e14156da0 2023-11-28T11:51:42.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.680+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e14140150 con 0x7f6e14156da0 2023-11-28T11:51:42.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.680+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1406b870 con 0x7f6e14156da0 2023-11-28T11:51:42.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.681+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e04025070 con 0x7f6e14156da0 2023-11-28T11:51:42.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.681+0000 7f6e09ffb700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6df80682f0 0x7f6df806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:42.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.681+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e04087380 con 0x7f6e14156da0 2023-11-28T11:51:42.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.681+0000 7f6e19793700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6df80682f0 0x7f6df806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:42.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.682+0000 7f6e19793700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6df80682f0 0x7f6df806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6e10000ef0 tx=0x7f6e10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:42.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.686+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e04053be0 con 0x7f6e14156da0 2023-11-28T11:51:42.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.871+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e14143a70 con 0x7f6e14156da0 2023-11-28T11:51:42.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.873+0000 7f6e09ffb700 1 -- 172.21.15.144:0/3398222255 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f6e04057200 con 0x7f6e14156da0 2023-11-28T11:51:42.873 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:42.874 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6df80682f0 msgr2=0x7f6df806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6df80682f0 0x7f6df806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6e10000ef0 tx=0x7f6e10009040 comp rx=0 tx=0).stop 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 msgr2=0x7f6e1413ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 0x7f6e1413ccf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e04000c00 tx=0x7f6e040047c0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 shutdown_connections 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6df80682f0 0x7f6df806a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e1413d230 0x7f6e1406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e00004850 0x7f6e14156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 --2- 172.21.15.144:0/3398222255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e14156da0 0x7f6e1413ccf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:42.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 >> 172.21.15.144:0/3398222255 conn(0x7f6e1406aed0 msgr2=0x7f6e1413db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:42.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 shutdown_connections 2023-11-28T11:51:42.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:42.875+0000 7f6e1b9f7700 1 -- 172.21.15.144:0/3398222255 wait complete. 2023-11-28T11:51:42.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:42.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:43.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.011+0000 7fa36f471700 1 Processor -- start 2023-11-28T11:51:43.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.011+0000 7fa36f471700 1 -- start start 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36f471700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cd8c0 0x7fa368140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36f471700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa3680cdc90 0x7fa3680b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36f471700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36813ff00 con 0x7fa3680b2e70 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36f471700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3681404c0 con 0x7fa3680cdc90 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36f471700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36815aa80 con 0x7fa3680cd8c0 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa3680cdc90 0x7fa3680b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa3680cdc90 0x7fa3680b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35356/0 (socket says 172.21.15.144:35356) 2023-11-28T11:51:43.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 -- 172.21.15.144:0/2710564887 learned_addr learned my addr 172.21.15.144:0/2710564887 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 -- 172.21.15.144:0/2710564887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cd8c0 msgr2=0x7fa368140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 --2- 172.21.15.144:0/2710564887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cd8c0 0x7fa368140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 -- 172.21.15.144:0/2710564887 >> v1:172.21.15.177:6789/0 conn(0x7fa3680b2e70 legacy=0x7fa36806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 -- 172.21.15.144:0/2710564887 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680b3310 con 0x7fa3680cdc90 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa36ca0c700 1 --2- 172.21.15.144:0/2710564887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680cdc90 0x7fa3680b2930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3580098f0 tx=0x7fa358009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88abfc8e83f6446d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa35ffff700 1 -- 172.21.15.144:0/2710564887 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358004230 con 0x7fa3680cdc90 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa35ffff700 1 -- 172.21.15.144:0/2710564887 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358004390 con 0x7fa3680cdc90 2023-11-28T11:51:43.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.012+0000 7fa35ffff700 1 -- 172.21.15.144:0/2710564887 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3580036a0 con 0x7fa3680cdc90 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 -- 172.21.15.144:0/2710564887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680cdc90 msgr2=0x7fa3680b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 --2- 172.21.15.144:0/2710564887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680cdc90 0x7fa3680b2930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3580098f0 tx=0x7fa358009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 -- 172.21.15.144:0/2710564887 shutdown_connections 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 --2- 172.21.15.144:0/2710564887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cd8c0 0x7fa368140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 --2- 172.21.15.144:0/2710564887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680cdc90 0x7fa3680b2930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 -- 172.21.15.144:0/2710564887 >> 172.21.15.144:0/2710564887 conn(0x7fa36806aed0 msgr2=0x7fa36806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 -- 172.21.15.144:0/2710564887 shutdown_connections 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.013+0000 7fa36f471700 1 -- 172.21.15.144:0/2710564887 wait complete. 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.014+0000 7fa36f471700 1 Processor -- start 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.014+0000 7fa36f471700 1 -- start start 2023-11-28T11:51:43.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36f471700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 0x7fa36815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36f471700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3680cd8c0 0x7fa3680b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36f471700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cdc90 0x7fa3680b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36f471700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680b4a20 con 0x7fa3680b2e70 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36f471700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681f0320 con 0x7fa3680cd8c0 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36f471700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680b3da0 con 0x7fa3680cdc90 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 0x7fa36815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 0x7fa36815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35360/0 (socket says 172.21.15.144:35360) 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 -- 172.21.15.144:0/2421733881 learned_addr learned my addr 172.21.15.144:0/2421733881 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36ca0c700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3680cd8c0 0x7fa3680b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36da0e700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cdc90 0x7fa3680b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 -- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cdc90 msgr2=0x7fa3680b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cdc90 0x7fa3680b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 -- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3680cd8c0 msgr2=0x7fa3680b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3680cd8c0 0x7fa3680b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 -- 172.21.15.144:0/2421733881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358009550 con 0x7fa3680b2e70 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.015+0000 7fa36d20d700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 0x7fa36815d8e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b550 tx=0x7fa36400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.016+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364005aa0 con 0x7fa3680b2e70 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.016+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364005c00 con 0x7fa3680b2e70 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.016+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa368140150 con 0x7fa3680b2e70 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.016+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364011630 con 0x7fa3680b2e70 2023-11-28T11:51:43.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.016+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3680b35f0 con 0x7fa3680b2e70 2023-11-28T11:51:43.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.017+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3680ce100 con 0x7fa3680b2e70 2023-11-28T11:51:43.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.017+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa36400f050 con 0x7fa3680b2e70 2023-11-28T11:51:43.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.017+0000 7fa35dffb700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa34c0682f0 0x7fa34c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.017+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3640857a0 con 0x7fa3680b2e70 2023-11-28T11:51:43.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.017+0000 7fa36ca0c700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa34c0682f0 0x7fa34c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.018+0000 7fa36ca0c700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa34c0682f0 0x7fa34c06a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3580098f0 tx=0x7fa358008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.022+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa364051fa0 con 0x7fa3680b2e70 2023-11-28T11:51:43.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.205+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa3680b4b60 con 0x7fa3680b2e70 2023-11-28T11:51:43.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.205+0000 7fa35dffb700 1 -- 172.21.15.144:0/2421733881 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa364015020 con 0x7fa3680b2e70 2023-11-28T11:51:43.206 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:43.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa34c0682f0 msgr2=0x7fa34c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa34c0682f0 0x7fa34c06a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3580098f0 tx=0x7fa358008030 comp rx=0 tx=0).stop 2023-11-28T11:51:43.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 msgr2=0x7fa36815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 0x7fa36815d8e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b550 tx=0x7fa36400b910 comp rx=0 tx=0).stop 2023-11-28T11:51:43.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 shutdown_connections 2023-11-28T11:51:43.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3680b2e70 0x7fa36815d8e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa34c0682f0 0x7fa34c06a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa3680cdc90 0x7fa3680b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 --2- 172.21.15.144:0/2421733881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3680cd8c0 0x7fa3680b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 >> 172.21.15.144:0/2421733881 conn(0x7fa36806aed0 msgr2=0x7fa3680d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:43.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 shutdown_connections 2023-11-28T11:51:43.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.207+0000 7fa36f471700 1 -- 172.21.15.144:0/2421733881 wait complete. 2023-11-28T11:51:43.220 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:43.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.341+0000 7ff80ef1d700 1 Processor -- start 2023-11-28T11:51:43.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.341+0000 7ff80ef1d700 1 -- start start 2023-11-28T11:51:43.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80ef1d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff8080b3040 0x7ff80815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80ef1d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80806b420 con 0x7ff8080ce040 2023-11-28T11:51:43.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80ef1d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80806b700 con 0x7ff8080d0a00 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80ef1d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80806ff60 con 0x7ff8080b3040 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80ccb9700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff8080ce040 0x7ff80806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52080/0 (socket says 172.21.15.144:52080) 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80ccb9700 1 -- 172.21.15.144:0/926318471 learned_addr learned my addr 172.21.15.144:0/926318471 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486767138 0 0) 0x7ff80806b420 con 0x7ff8080ce040 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/926318471 >> v2:172.21.15.177:3300/0 conn(0x7ff8080b3040 0x7ff80815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7f80035b0 con 0x7ff8080ce040 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664269742 0 0) 0x7ff7f80035b0 con 0x7ff8080ce040 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 >> v2:172.21.15.177:3300/0 conn(0x7ff8080b3040 msgr2=0x7ff80815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 --2- 172.21.15.144:0/926318471 >> v2:172.21.15.177:3300/0 conn(0x7ff8080b3040 0x7ff80815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 >> v1:172.21.15.177:6789/0 conn(0x7ff8080d0a00 legacy=0x7ff8080b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80806b9e0 con 0x7ff8080ce040 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7ff7fc0017f0 con 0x7ff8080ce040 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 >> v1:172.21.15.144:6789/0 conn(0x7ff8080ce040 legacy=0x7ff80806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8003a00 0x7ff7f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f8005200 0x7ff7f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80806ff60 con 0x7ff7f8005200 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f800c080 con 0x7ff7f8004850 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f800c240 con 0x7ff7f8003a00 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.342+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f8005200 0x7ff7f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80ccb9700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8003a00 0x7ff7f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff807fff700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 -- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8003a00 msgr2=0x7ff7f8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8003a00 0x7ff7f8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 -- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8004850 msgr2=0x7ff7f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 -- 172.21.15.144:0/926318471 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80035b0 con 0x7ff7f8005200 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff807fff700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8004850 0x7ff7f8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 -- 172.21.15.144:0/926318471 reap_dead start 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f8005200 0x7ff7f800ba10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff800009f90 tx=0x7ff7f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda8fb89cb9323a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800003a20 con 0x7ff7f8005200 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000f040 con 0x7ff7f8005200 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff8077fe700 1 -- 172.21.15.144:0/926318471 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800003de0 con 0x7ff7f8005200 2023-11-28T11:51:43.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80ef1d700 1 -- 172.21.15.144:0/926318471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f8005200 msgr2=0x7ff7f800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.343+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f8005200 0x7ff7f800ba10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff800009f90 tx=0x7ff7f8005880 comp rx=0 tx=0).stop 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.344+0000 7ff80ef1d700 1 -- 172.21.15.144:0/926318471 shutdown_connections 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.344+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/926318471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f8005200 0x7ff7f800ba10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.344+0000 7ff80ef1d700 1 -- 172.21.15.144:0/926318471 >> 172.21.15.144:0/926318471 conn(0x7ff80806aed0 msgr2=0x7ff80806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.344+0000 7ff80ef1d700 1 -- 172.21.15.144:0/926318471 shutdown_connections 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.344+0000 7ff80ef1d700 1 -- 172.21.15.144:0/926318471 wait complete. 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.345+0000 7ff80ef1d700 1 Processor -- start 2023-11-28T11:51:43.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.345+0000 7ff80ef1d700 1 -- start start 2023-11-28T11:51:43.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8005200 0x7ff8081f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 0x7ff8080b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8081f5eb0 0x7ff8080cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080b4ad0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081f5a20 con 0x7ff8081f5eb0 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081f1e60 con 0x7ff7f8005200 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 0x7ff8080b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 0x7ff8080b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35382/0 (socket says 172.21.15.144:35382) 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 -- 172.21.15.144:0/3153256714 learned_addr learned my addr 172.21.15.144:0/3153256714 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8081f5eb0 0x7ff8080cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ccb9700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8005200 0x7ff8081f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 -- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8005200 msgr2=0x7ff8081f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8005200 0x7ff8081f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 -- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8081f5eb0 msgr2=0x7ff8080cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8081f5eb0 0x7ff8080cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 -- 172.21.15.144:0/3153256714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80806bfa0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80d4ba700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8081f5eb0 0x7ff8080cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff807fff700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 0x7ff8080b3d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400e4c0 tx=0x7ff7f400e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40052e0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4005440 con 0x7ff8080b6ab0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40038c0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80806bcc0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.346+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80815df10 con 0x7ff8080b6ab0 2023-11-28T11:51:43.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.347+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff808068690 con 0x7ff8080b6ab0 2023-11-28T11:51:43.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.348+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff7f401b030 con 0x7ff8080b6ab0 2023-11-28T11:51:43.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.348+0000 7ff8057fa700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7ec068320 0x7ff7ec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.348+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff7f4085070 con 0x7ff8080b6ab0 2023-11-28T11:51:43.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.348+0000 7ff80ccb9700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7ec068320 0x7ff7ec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.348+0000 7ff80ccb9700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7ec068320 0x7ff7ec06a7d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0011d0 tx=0x7ff7fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.352+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7f404d5e0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.533+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8080b4510 con 0x7ff8080b6ab0 2023-11-28T11:51:43.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.534+0000 7ff8057fa700 1 -- 172.21.15.144:0/3153256714 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7f40190a0 con 0x7ff8080b6ab0 2023-11-28T11:51:43.535 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7ec068320 msgr2=0x7ff7ec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7ec068320 0x7ff7ec06a7d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0011d0 tx=0x7ff7fc009040 comp rx=0 tx=0).stop 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 msgr2=0x7ff8080b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 0x7ff8080b3d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400e4c0 tx=0x7ff7f400e880 comp rx=0 tx=0).stop 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 shutdown_connections 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7ec068320 0x7ff7ec06a7d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8081f5eb0 0x7ff8080cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7f8005200 0x7ff8081f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 --2- 172.21.15.144:0/3153256714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8080b6ab0 0x7ff8080b3d00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 >> 172.21.15.144:0/3153256714 conn(0x7ff80806aed0 msgr2=0x7ff80815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:43.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 shutdown_connections 2023-11-28T11:51:43.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.536+0000 7ff80ef1d700 1 -- 172.21.15.144:0/3153256714 wait complete. 2023-11-28T11:51:43.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:43.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.672+0000 7f4e5126e700 1 Processor -- start 2023-11-28T11:51:43.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.672+0000 7f4e5126e700 1 -- start start 2023-11-28T11:51:43.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e5126e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e4c15ca50 0x7f4e4c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e5126e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0cf5e0 con 0x7f4e4c0cce10 2023-11-28T11:51:43.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e5126e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0cecb0 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e5126e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c06bd60 con 0x7f4e4c15ca50 2023-11-28T11:51:43.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e4c15ca50 0x7f4e4c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e4c15ca50 0x7f4e4c06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35388/0 (socket says 172.21.15.144:35388) 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 -- 172.21.15.144:0/2251354711 learned_addr learned my addr 172.21.15.144:0/2251354711 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 -- 172.21.15.144:0/2251354711 >> v1:172.21.15.177:6789/0 conn(0x7f4e4c06b8c0 legacy=0x7f4e4c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 -- 172.21.15.144:0/2251354711 >> v1:172.21.15.144:6789/0 conn(0x7f4e4c0cce10 legacy=0x7f4e4c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 -- 172.21.15.144:0/2251354711 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0cef90 con 0x7f4e4c15ca50 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e49ffb700 1 -- 172.21.15.144:0/2251354711 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195176740 0 0) 0x7f4e4c0cf5e0 con 0x7f4e4c0cce10 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.673+0000 7f4e4affd700 1 --2- 172.21.15.144:0/2251354711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c15ca50 0x7f4e4c06b380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e400098f0 tx=0x7f4e40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31910e7d8e591eeb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e49ffb700 1 -- 172.21.15.144:0/2251354711 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40004230 con 0x7f4e4c15ca50 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e49ffb700 1 -- 172.21.15.144:0/2251354711 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40004390 con 0x7f4e4c15ca50 2023-11-28T11:51:43.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e49ffb700 1 -- 172.21.15.144:0/2251354711 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400036a0 con 0x7f4e4c15ca50 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 -- 172.21.15.144:0/2251354711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c15ca50 msgr2=0x7f4e4c06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2251354711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c15ca50 0x7f4e4c06b380 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e400098f0 tx=0x7f4e40009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 -- 172.21.15.144:0/2251354711 shutdown_connections 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2251354711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c15ca50 0x7f4e4c06b380 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 -- 172.21.15.144:0/2251354711 >> 172.21.15.144:0/2251354711 conn(0x7f4e4c09a230 msgr2=0x7f4e4c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 -- 172.21.15.144:0/2251354711 shutdown_connections 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.674+0000 7f4e5126e700 1 -- 172.21.15.144:0/2251354711 wait complete. 2023-11-28T11:51:43.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.675+0000 7f4e5126e700 1 Processor -- start 2023-11-28T11:51:43.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 -- start start 2023-11-28T11:51:43.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 0x7f4e4c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c0cce10 0x7f4e4c15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c15ca50 0x7f4e4c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15e650 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15e8f0 con 0x7f4e4c0cce10 2023-11-28T11:51:43.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e5126e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15eb90 con 0x7f4e4c15ca50 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 0x7f4e4c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 0x7f4e4c151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35394/0 (socket says 172.21.15.144:35394) 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e4affd700 1 -- 172.21.15.144:0/2677544860 learned_addr learned my addr 172.21.15.144:0/2677544860 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e4a7fc700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c0cce10 0x7f4e4c15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.676+0000 7f4e4b7fe700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c15ca50 0x7f4e4c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4affd700 1 -- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c15ca50 msgr2=0x7f4e4c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4affd700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c15ca50 0x7f4e4c0b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4affd700 1 -- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c0cce10 msgr2=0x7f4e4c15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4affd700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c0cce10 0x7f4e4c15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4affd700 1 -- 172.21.15.144:0/2677544860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009550 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4a7fc700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c0cce10 0x7f4e4c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4b7fe700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c15ca50 0x7f4e4c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e4affd700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 0x7f4e4c151b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c06d870 tx=0x7f4e40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000b660 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4000b7e0 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c15ce20 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000bb90 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.677+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c0cf270 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.678+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c15cfb0 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.678+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e40017020 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.679+0000 7f4e337fe700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4e380682f0 0x7f4e3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.679+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e40087650 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.679+0000 7f4e4a7fc700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4e380682f0 0x7f4e3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.679+0000 7f4e4a7fc700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4e380682f0 0x7f4e3806a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c1f59f0 tx=0x7f4e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:43.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.683+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e40053d60 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.867+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e4c0d0f00 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.867+0000 7f4e337fe700 1 -- 172.21.15.144:0/2677544860 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e4000bcf0 con 0x7f4e4c06b8c0 2023-11-28T11:51:43.868 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:43.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4e380682f0 msgr2=0x7f4e3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4e380682f0 0x7f4e3806a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c1f59f0 tx=0x7f4e3c009040 comp rx=0 tx=0).stop 2023-11-28T11:51:43.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 msgr2=0x7f4e4c151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 0x7f4e4c151b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c06d870 tx=0x7f4e40004750 comp rx=0 tx=0).stop 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 shutdown_connections 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e4c06b8c0 0x7f4e4c151b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4e380682f0 0x7f4e3806a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4e4c15ca50 0x7f4e4c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 --2- 172.21.15.144:0/2677544860 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e4c0cce10 0x7f4e4c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 >> 172.21.15.144:0/2677544860 conn(0x7f4e4c09a230 msgr2=0x7f4e4c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 shutdown_connections 2023-11-28T11:51:43.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:43.869+0000 7f4e5126e700 1 -- 172.21.15.144:0/2677544860 wait complete. 2023-11-28T11:51:43.882 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:44.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.004+0000 7f3bdc8e3700 1 Processor -- start 2023-11-28T11:51:44.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.004+0000 7f3bdc8e3700 1 -- start start 2023-11-28T11:51:44.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bdc8e3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3bd806dee0 0x7f3bd806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bdc8e3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bdc8e3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd815d060 con 0x7f3bd8071040 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bdc8e3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd815d340 con 0x7f3bd806dee0 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bdc8e3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd815d620 con 0x7f3bd8160540 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bd5d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3bd806dee0 0x7f3bd806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bd5d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3bd806dee0 0x7f3bd806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35404/0 (socket says 172.21.15.144:35404) 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.005+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/3977534715 learned_addr learned my addr 172.21.15.144:0/3977534715 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/3977534715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 msgr2=0x7f3bd81609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd6d9d700 1 --2- 172.21.15.144:0/3977534715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/3977534715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd81609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/3977534715 >> v1:172.21.15.177:6789/0 conn(0x7f3bd8071040 legacy=0x7f3bd8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/3977534715 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd815d900 con 0x7f3bd806dee0 2023-11-28T11:51:44.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/3977534715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd806e330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80098f0 tx=0x7f3bc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9748c294f69ce4cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd559a700 1 -- 172.21.15.144:0/3977534715 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004230 con 0x7f3bd806dee0 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd559a700 1 -- 172.21.15.144:0/3977534715 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc8004390 con 0x7f3bd806dee0 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bd559a700 1 -- 172.21.15.144:0/3977534715 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80036a0 con 0x7f3bd806dee0 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/3977534715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 msgr2=0x7f3bd806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.006+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/3977534715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd806e330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80098f0 tx=0x7f3bc8009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.007+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/3977534715 shutdown_connections 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.007+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/3977534715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.007+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/3977534715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd806e330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.007+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/3977534715 >> 172.21.15.144:0/3977534715 conn(0x7f3bd80d0000 msgr2=0x7f3bd80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.007+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/3977534715 shutdown_connections 2023-11-28T11:51:44.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.007+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/3977534715 wait complete. 2023-11-28T11:51:44.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.008+0000 7f3bdc8e3700 1 Processor -- start 2023-11-28T11:51:44.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.008+0000 7f3bdc8e3700 1 -- start start 2023-11-28T11:51:44.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd815b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071040 0x7f3bd8145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80c5490 con 0x7f3bd806dee0 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8158e10 con 0x7f3bd8071040 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd815bc40 con 0x7f3bd8160540 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35408/0 (socket says 172.21.15.144:35408) 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 -- 172.21.15.144:0/1192889165 learned_addr learned my addr 172.21.15.144:0/1192889165 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd6d9d700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071040 0x7f3bd8145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 -- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 msgr2=0x7f3bd8158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd8158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 -- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071040 msgr2=0x7f3bd8145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071040 0x7f3bd8145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 -- 172.21.15.144:0/1192889165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009550 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071040 0x7f3bd8145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bd659c700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd815b700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00b4a0 tx=0x7f3bcc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc011830 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc011990 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd815ccd0 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.009+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc011d40 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.010+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd806e870 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.011+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd806a270 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.011+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3bcc00f050 con 0x7f3bd806dee0 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.011+0000 7f3bc6ffd700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.011+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3bcc085800 con 0x7f3bd806dee0 2023-11-28T11:51:44.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.011+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.012+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009840 tx=0x7f3bc8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.015+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3bcc052060 con 0x7f3bd806dee0 2023-11-28T11:51:44.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.199+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bd806bad0 con 0x7f3bd806dee0 2023-11-28T11:51:44.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.199+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/1192889165 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bcc055680 con 0x7f3bd806dee0 2023-11-28T11:51:44.200 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:44.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 msgr2=0x7f3bb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009840 tx=0x7f3bc8008030 comp rx=0 tx=0).stop 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 msgr2=0x7f3bd815b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd815b700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00b4a0 tx=0x7f3bcc00b860 comp rx=0 tx=0).stop 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 shutdown_connections 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bd806dee0 0x7f3bd815b700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd8160540 0x7f3bd8158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 --2- 172.21.15.144:0/1192889165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071040 0x7f3bd8145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 >> 172.21.15.144:0/1192889165 conn(0x7f3bd80d0000 msgr2=0x7f3bd80aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 shutdown_connections 2023-11-28T11:51:44.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.201+0000 7f3bdc8e3700 1 -- 172.21.15.144:0/1192889165 wait complete. 2023-11-28T11:51:44.222 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:51:44.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:51:44.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.342+0000 7fb14a762700 1 Processor -- start 2023-11-28T11:51:44.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.342+0000 7fb14a762700 1 -- start start 2023-11-28T11:51:44.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb14a762700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb144071040 0x7fb144071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb14a762700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb144071950 con 0x7fb14406dee0 2023-11-28T11:51:44.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb14a762700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14415d060 con 0x7fb144160540 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb14a762700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14415d340 con 0x7fb144071040 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb144071040 0x7fb144071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb144071040 0x7fb144071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35416/0 (socket says 172.21.15.144:35416) 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 -- 172.21.15.144:0/2575560451 learned_addr learned my addr 172.21.15.144:0/2575560451 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 -- 172.21.15.144:0/2575560451 >> v1:172.21.15.177:6789/0 conn(0x7fb144160540 legacy=0x7fb144160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 -- 172.21.15.144:0/2575560451 >> v1:172.21.15.144:6789/0 conn(0x7fb14406dee0 legacy=0x7fb14406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 -- 172.21.15.144:0/2575560451 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14415d620 con 0x7fb144071040 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.343+0000 7fb143fff700 1 --2- 172.21.15.144:0/2575560451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144071040 0x7fb144071410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0098f0 tx=0x7fb12c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dcb034bdf98dab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb142ffd700 1 -- 172.21.15.144:0/2575560451 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c004230 con 0x7fb144071040 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb142ffd700 1 -- 172.21.15.144:0/2575560451 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c004390 con 0x7fb144071040 2023-11-28T11:51:44.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb142ffd700 1 -- 172.21.15.144:0/2575560451 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0036a0 con 0x7fb144071040 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 -- 172.21.15.144:0/2575560451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144071040 msgr2=0x7fb144071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 --2- 172.21.15.144:0/2575560451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144071040 0x7fb144071410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0098f0 tx=0x7fb12c009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 -- 172.21.15.144:0/2575560451 shutdown_connections 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 --2- 172.21.15.144:0/2575560451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144071040 0x7fb144071410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 -- 172.21.15.144:0/2575560451 >> 172.21.15.144:0/2575560451 conn(0x7fb1440d0000 msgr2=0x7fb1440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 -- 172.21.15.144:0/2575560451 shutdown_connections 2023-11-28T11:51:44.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.344+0000 7fb14a762700 1 -- 172.21.15.144:0/2575560451 wait complete. 2023-11-28T11:51:44.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.346+0000 7fb14a762700 1 Processor -- start 2023-11-28T11:51:44.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.346+0000 7fb14a762700 1 -- start start 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb14a762700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb14406dee0 0x7fb1440c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb14a762700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb144071040 0x7fb1440ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb14a762700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 0x7fb1441f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb14a762700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1441f7b40 con 0x7fb144160540 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb14a762700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1441f7e20 con 0x7fb144071040 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb14a762700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440c3aa0 con 0x7fb14406dee0 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 0x7fb1441f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 0x7fb1441f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35426/0 (socket says 172.21.15.144:35426) 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 -- 172.21.15.144:0/2408194248 learned_addr learned my addr 172.21.15.144:0/2408194248 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb1437fe700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb144071040 0x7fb1440ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb143fff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb14406dee0 0x7fb1440c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 -- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb14406dee0 msgr2=0x7fb1440c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb14406dee0 0x7fb1440c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 -- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb144071040 msgr2=0x7fb1440ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb144071040 0x7fb1440ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 -- 172.21.15.144:0/2408194248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c009550 con 0x7fb144160540 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb143fff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb14406dee0 0x7fb1440c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.347+0000 7fb148cff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 0x7fb1441f6ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b990 tx=0x7fb13c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.348+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00c9d0 con 0x7fb144160540 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.348+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c00cb30 con 0x7fb144160540 2023-11-28T11:51:44.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.348+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14415ccd0 con 0x7fb144160540 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.348+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c014440 con 0x7fb144160540 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.348+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14415d900 con 0x7fb144160540 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.349+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14406a270 con 0x7fb144160540 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.349+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb13c01b030 con 0x7fb144160540 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.349+0000 7fb140ff9700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1240682f0 0x7fb12406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.350+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb13c085900 con 0x7fb144160540 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.350+0000 7fb143fff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1240682f0 0x7fb12406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.350+0000 7fb143fff700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1240682f0 0x7fb12406a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009c40 tx=0x7fb12c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.354+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb13c052160 con 0x7fb144160540 2023-11-28T11:51:44.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.607+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fb1440720a0 con 0x7fb144160540 2023-11-28T11:51:44.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.735+0000 7fb140ff9700 1 -- 172.21.15.144:0/2408194248 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v45) v1 ==== 145+0+0 (secure 0 0 0) 0x7fb13c0190d0 con 0x7fb144160540 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1240682f0 msgr2=0x7fb12406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1240682f0 0x7fb12406a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009c40 tx=0x7fb12c008030 comp rx=0 tx=0).stop 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 msgr2=0x7fb1441f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 0x7fb1441f6ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b990 tx=0x7fb13c00bd50 comp rx=0 tx=0).stop 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 shutdown_connections 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb144160540 0x7fb1441f6ff0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb1240682f0 0x7fb12406a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb14406dee0 0x7fb1440c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 --2- 172.21.15.144:0/2408194248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb144071040 0x7fb1440ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 >> 172.21.15.144:0/2408194248 conn(0x7fb1440d0000 msgr2=0x7fb1440cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:44.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 shutdown_connections 2023-11-28T11:51:44.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.737+0000 7fb14a762700 1 -- 172.21.15.144:0/2408194248 wait complete. 2023-11-28T11:51:44.750 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:51:44.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.872+0000 7f97d362e700 1 Processor -- start 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.873+0000 7f97d362e700 1 -- start start 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.873+0000 7f97d362e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.873+0000 7f97d362e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97cc0b2e70 0x7f97cc06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.873+0000 7f97d362e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc13ff00 con 0x7f97cc0cd8c0 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.873+0000 7f97d362e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc1404c0 con 0x7f97cc0b2e70 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.873+0000 7f97d362e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc140dc0 con 0x7f97cc140950 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97cc0b2e70 0x7f97cc06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97cc0b2e70 0x7f97cc06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35434/0 (socket says 172.21.15.144:35434) 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 -- 172.21.15.144:0/2933766619 learned_addr learned my addr 172.21.15.144:0/2933766619 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d13ca700 1 --1- 172.21.15.144:0/2933766619 >> v1:172.21.15.144:6789/0 conn(0x7f97cc0cd8c0 0x7f97cc0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52142/0 (socket says 172.21.15.144:52142) 2023-11-28T11:51:44.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d0bc9700 1 --2- 172.21.15.144:0/2933766619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 -- 172.21.15.144:0/2933766619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 msgr2=0x7f97cc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 --2- 172.21.15.144:0/2933766619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 -- 172.21.15.144:0/2933766619 >> v1:172.21.15.144:6789/0 conn(0x7f97cc0cd8c0 legacy=0x7f97cc0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 -- 172.21.15.144:0/2933766619 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc15aa80 con 0x7f97cc0b2e70 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97d1bcb700 1 --2- 172.21.15.144:0/2933766619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc06f850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009e50 tx=0x7f97c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578fce77663a3575 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97c3fff700 1 -- 172.21.15.144:0/2933766619 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400e070 con 0x7f97cc0b2e70 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97c3fff700 1 -- 172.21.15.144:0/2933766619 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400b9c0 con 0x7f97cc0b2e70 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.874+0000 7f97c3fff700 1 -- 172.21.15.144:0/2933766619 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400bd70 con 0x7f97cc0b2e70 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 -- 172.21.15.144:0/2933766619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 msgr2=0x7f97cc06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 --2- 172.21.15.144:0/2933766619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc06f850 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009e50 tx=0x7f97c400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 -- 172.21.15.144:0/2933766619 shutdown_connections 2023-11-28T11:51:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 --2- 172.21.15.144:0/2933766619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc06f850 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 --2- 172.21.15.144:0/2933766619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 -- 172.21.15.144:0/2933766619 >> 172.21.15.144:0/2933766619 conn(0x7f97cc06aed0 msgr2=0x7f97cc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 -- 172.21.15.144:0/2933766619 shutdown_connections 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.875+0000 7f97d362e700 1 -- 172.21.15.144:0/2933766619 wait complete. 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.876+0000 7f97d362e700 1 Processor -- start 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.876+0000 7f97d362e700 1 -- start start 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97cc0cd8c0 0x7f97cc0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0c4540 con 0x7f97cc0b2e70 2023-11-28T11:51:44.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0b3d00 con 0x7f97cc0cd8c0 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0c3a90 con 0x7f97cc140950 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc0c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35436/0 (socket says 172.21.15.144:35436) 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 -- 172.21.15.144:0/3899945808 learned_addr learned my addr 172.21.15.144:0/3899945808 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d0bc9700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97cc0cd8c0 0x7f97cc0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d1bcb700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 -- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 msgr2=0x7f97cc0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 -- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97cc0cd8c0 msgr2=0x7f97cc0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97cc0cd8c0 0x7f97cc0b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 -- 172.21.15.144:0/3899945808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4009b30 con 0x7f97cc0b2e70 2023-11-28T11:51:44.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d13ca700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc0c3550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c500 tx=0x7f97bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0074e0 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc007640 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc0b35f0 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.877+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc005a70 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.878+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc0b3310 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.878+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc140150 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.879+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97bc019030 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.879+0000 7f97c1ffb700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f97b00682f0 0x7f97b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.879+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97bc084fe0 con 0x7f97cc0b2e70 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.879+0000 7f97d0bc9700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f97b00682f0 0x7f97b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.880+0000 7f97d0bc9700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f97b00682f0 0x7f97b006a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0cf240 tx=0x7f97c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:44.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:44.883+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97bc051840 con 0x7f97cc0b2e70 2023-11-28T11:51:45.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.070+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f97cc068690 con 0x7f97cc0b2e70 2023-11-28T11:51:45.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.073+0000 7f97c1ffb700 1 -- 172.21.15.144:0/3899945808 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f97bc0170c0 con 0x7f97cc0b2e70 2023-11-28T11:51:45.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f97b00682f0 msgr2=0x7f97b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f97b00682f0 0x7f97b006a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0cf240 tx=0x7f97c8008040 comp rx=0 tx=0).stop 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 msgr2=0x7f97cc0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc0c3550 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c500 tx=0x7f97bc00c8c0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 shutdown_connections 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97cc0b2e70 0x7f97cc0c3550 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.075+0000 7f97d362e700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f97b00682f0 0x7f97b006a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.076+0000 7f97d362e700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97cc140950 0x7f97cc0b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.076+0000 7f97d362e700 1 --2- 172.21.15.144:0/3899945808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97cc0cd8c0 0x7f97cc0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.076+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 >> 172.21.15.144:0/3899945808 conn(0x7f97cc06aed0 msgr2=0x7f97cc13e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.076+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 shutdown_connections 2023-11-28T11:51:45.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.076+0000 7f97d362e700 1 -- 172.21.15.144:0/3899945808 wait complete. 2023-11-28T11:51:45.088 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:51:45.089 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:51:45.132 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:51:45.182 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:51:45.182 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:45.183 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:45.200 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:45.200 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:45.257 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:45.257 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:45.315 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:45.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.474+0000 7fb4ab1b3700 1 Processor -- start 2023-11-28T11:51:45.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.474+0000 7fb4ab1b3700 1 -- start start 2023-11-28T11:51:45.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4ab1b3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4a4071040 0x7fb4a4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4ab1b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406dee0 0x7fb4a406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4ab1b3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a4071950 con 0x7fb4a406cdc0 2023-11-28T11:51:45.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4ab1b3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a4161550 con 0x7fb4a4071040 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4ab1b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a415d060 con 0x7fb4a406dee0 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4a4071040 0x7fb4a4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4a4071040 0x7fb4a4071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35444/0 (socket says 172.21.15.144:35444) 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 -- 172.21.15.144:0/1265776534 learned_addr learned my addr 172.21.15.144:0/1265776534 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a9750700 1 --1- 172.21.15.144:0/1265776534 >> v1:172.21.15.144:6789/0 conn(0x7fb4a406cdc0 0x7fb4a406d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52156/0 (socket says 172.21.15.144:52156) 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a3fff700 1 --2- 172.21.15.144:0/1265776534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406dee0 0x7fb4a406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 -- 172.21.15.144:0/1265776534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406dee0 msgr2=0x7fb4a406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 --2- 172.21.15.144:0/1265776534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406dee0 0x7fb4a406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 -- 172.21.15.144:0/1265776534 >> v1:172.21.15.144:6789/0 conn(0x7fb4a406cdc0 legacy=0x7fb4a406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 -- 172.21.15.144:0/1265776534 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a415d340 con 0x7fb4a4071040 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.475+0000 7fb4a8f4f700 1 --2- 172.21.15.144:0/1265776534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a4071040 0x7fb4a4071410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb4980098f0 tx=0x7fb498009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ba6320c89af2e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4a37fe700 1 -- 172.21.15.144:0/1265776534 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49800b8a0 con 0x7fb4a4071040 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4a37fe700 1 -- 172.21.15.144:0/1265776534 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49800ba00 con 0x7fb4a4071040 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4a37fe700 1 -- 172.21.15.144:0/1265776534 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49800bd90 con 0x7fb4a4071040 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/1265776534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a4071040 msgr2=0x7fb4a4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/1265776534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a4071040 0x7fb4a4071410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb4980098f0 tx=0x7fb498009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/1265776534 shutdown_connections 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/1265776534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a4071040 0x7fb4a4071410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/1265776534 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406dee0 0x7fb4a406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/1265776534 >> 172.21.15.144:0/1265776534 conn(0x7fb4a40d0000 msgr2=0x7fb4a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:45.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/1265776534 shutdown_connections 2023-11-28T11:51:45.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.476+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/1265776534 wait complete. 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.477+0000 7fb4ab1b3700 1 Processor -- start 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.477+0000 7fb4ab1b3700 1 -- start start 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4ab1b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406cdc0 0x7fb4a415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4ab1b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 0x7fb4a40c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4ab1b3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4a4071040 0x7fb4a40c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4ab1b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40b3d30 con 0x7fb4a406dee0 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4ab1b3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40c6a40 con 0x7fb4a4071040 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4ab1b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40c6d20 con 0x7fb4a406cdc0 2023-11-28T11:51:45.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 0x7fb4a40c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4a3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 0x7fb4a40c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35450/0 (socket says 172.21.15.144:35450) 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.478+0000 7fb4a3fff700 1 -- 172.21.15.144:0/2671317648 learned_addr learned my addr 172.21.15.144:0/2671317648 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a8f4f700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406cdc0 0x7fb4a415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a9750700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4a4071040 0x7fb4a40c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a3fff700 1 -- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406cdc0 msgr2=0x7fb4a415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a3fff700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406cdc0 0x7fb4a415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a3fff700 1 -- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4a4071040 msgr2=0x7fb4a40c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a3fff700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4a4071040 0x7fb4a40c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a3fff700 1 -- 172.21.15.144:0/2671317648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb498009550 con 0x7fb4a406dee0 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a9750700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4a4071040 0x7fb4a40c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a3fff700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 0x7fb4a40c5cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d800 tx=0x7fb49000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:45.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900125f0 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb490012750 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490012b00 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a415d900 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.479+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a415d620 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.480+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a406e870 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.480+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb490012c60 con 0x7fb4a406dee0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.481+0000 7fb4a17fa700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4880682f0 0x7fb48806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.481+0000 7fb4a8f4f700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4880682f0 0x7fb48806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.481+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb490086180 con 0x7fb4a406dee0 2023-11-28T11:51:45.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.481+0000 7fb4a8f4f700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4880682f0 0x7fb48806a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb498008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:45.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.485+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb490052950 con 0x7fb4a406dee0 2023-11-28T11:51:45.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.669+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4a40720a0 con 0x7fb4a406dee0 2023-11-28T11:51:45.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.670+0000 7fb4a17fa700 1 -- 172.21.15.144:0/2671317648 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fb490055f70 con 0x7fb4a406dee0 2023-11-28T11:51:45.671 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:45.671 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:45.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4880682f0 msgr2=0x7fb48806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4880682f0 0x7fb48806a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb498008030 comp rx=0 tx=0).stop 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 msgr2=0x7fb4a40c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 0x7fb4a40c5cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d800 tx=0x7fb49000db10 comp rx=0 tx=0).stop 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 shutdown_connections 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb4880682f0 0x7fb48806a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4a4071040 0x7fb4a40c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb4a406cdc0 0x7fb4a415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 --2- 172.21.15.144:0/2671317648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4a406dee0 0x7fb4a40c5cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 >> 172.21.15.144:0/2671317648 conn(0x7fb4a40d0000 msgr2=0x7fb4a40cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 shutdown_connections 2023-11-28T11:51:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.672+0000 7fb4ab1b3700 1 -- 172.21.15.144:0/2671317648 wait complete. 2023-11-28T11:51:45.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:45.685 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:45.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.809+0000 7f9c7adad700 1 Processor -- start 2023-11-28T11:51:45.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.809+0000 7f9c7adad700 1 -- start start 2023-11-28T11:51:45.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7adad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c7406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7adad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c74160540 0x7f9c741609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7adad700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c7415d060 con 0x7f9c74071040 2023-11-28T11:51:45.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7adad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7415d340 con 0x7f9c74160540 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7adad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7415d620 con 0x7f9c7406dee0 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c74160540 0x7f9c741609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c74160540 0x7f9c741609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35460/0 (socket says 172.21.15.144:35460) 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 -- 172.21.15.144:0/3063004024 learned_addr learned my addr 172.21.15.144:0/3063004024 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c73fff700 1 --2- 172.21.15.144:0/3063004024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c7406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 -- 172.21.15.144:0/3063004024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 msgr2=0x7f9c7406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 --2- 172.21.15.144:0/3063004024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c7406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 -- 172.21.15.144:0/3063004024 >> v1:172.21.15.177:6789/0 conn(0x7f9c74071040 legacy=0x7f9c74071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.810+0000 7f9c7934a700 1 -- 172.21.15.144:0/3063004024 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7415d900 con 0x7f9c74160540 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7934a700 1 --2- 172.21.15.144:0/3063004024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74160540 0x7f9c741609c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009e50 tx=0x7f9c6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2040e5c408565b0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c737fe700 1 -- 172.21.15.144:0/3063004024 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00e070 con 0x7f9c74160540 2023-11-28T11:51:45.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c737fe700 1 -- 172.21.15.144:0/3063004024 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c00b9c0 con 0x7f9c74160540 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c737fe700 1 -- 172.21.15.144:0/3063004024 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c00bd70 con 0x7f9c74160540 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 -- 172.21.15.144:0/3063004024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74160540 msgr2=0x7f9c741609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 --2- 172.21.15.144:0/3063004024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74160540 0x7f9c741609c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009e50 tx=0x7f9c6c00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 -- 172.21.15.144:0/3063004024 shutdown_connections 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 --2- 172.21.15.144:0/3063004024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74160540 0x7f9c741609c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 --2- 172.21.15.144:0/3063004024 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c7406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 -- 172.21.15.144:0/3063004024 >> 172.21.15.144:0/3063004024 conn(0x7f9c740d0000 msgr2=0x7f9c740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 -- 172.21.15.144:0/3063004024 shutdown_connections 2023-11-28T11:51:45.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.811+0000 7f9c7adad700 1 -- 172.21.15.144:0/3063004024 wait complete. 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.812+0000 7f9c7adad700 1 Processor -- start 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 -- start start 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c740b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 0x7f9c740c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c74160540 0x7f9c74152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c5c10 con 0x7f9c74071040 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740b3530 con 0x7f9c74160540 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c7adad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740b3810 con 0x7f9c7406dee0 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 0x7f9c740c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 0x7f9c740c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35462/0 (socket says 172.21.15.144:35462) 2023-11-28T11:51:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.813+0000 7f9c73fff700 1 -- 172.21.15.144:0/322049604 learned_addr learned my addr 172.21.15.144:0/322049604 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c78b49700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c740b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c7934a700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c74160540 0x7f9c74152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c73fff700 1 -- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 msgr2=0x7f9c740b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c73fff700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c740b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c73fff700 1 -- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c74160540 msgr2=0x7f9c74152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c73fff700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c74160540 0x7f9c74152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c73fff700 1 -- 172.21.15.144:0/322049604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c009b30 con 0x7f9c74071040 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c7934a700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c74160540 0x7f9c74152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c73fff700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 0x7f9c740c56d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000d800 tx=0x7f9c6000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600125f0 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c60012750 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60012b00 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7415ccd0 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.814+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7406e870 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.815+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7406a270 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.815+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c60012c60 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.816+0000 7f9c717fa700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c580682f0 0x7f9c5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.816+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c60085f20 con 0x7f9c74071040 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.816+0000 7f9c78b49700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c580682f0 0x7f9c5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.816+0000 7f9c78b49700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c580682f0 0x7f9c5806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9c68007fd0 tx=0x7f9c68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:45.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:45.820+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c60052830 con 0x7f9c74071040 2023-11-28T11:51:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.002+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9c740c6e00 con 0x7f9c74071040 2023-11-28T11:51:46.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.002+0000 7f9c717fa700 1 -- 172.21.15.144:0/322049604 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9c60055e50 con 0x7f9c74071040 2023-11-28T11:51:46.003 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c580682f0 msgr2=0x7f9c5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c580682f0 0x7f9c5806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9c68007fd0 tx=0x7f9c68009040 comp rx=0 tx=0).stop 2023-11-28T11:51:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 msgr2=0x7f9c740c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 0x7f9c740c56d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000d800 tx=0x7f9c6000db10 comp rx=0 tx=0).stop 2023-11-28T11:51:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 shutdown_connections 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c580682f0 0x7f9c5806a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c74160540 0x7f9c74152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c7406dee0 0x7f9c740b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 --2- 172.21.15.144:0/322049604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c74071040 0x7f9c740c56d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 >> 172.21.15.144:0/322049604 conn(0x7f9c740d0000 msgr2=0x7f9c74071410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 shutdown_connections 2023-11-28T11:51:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.004+0000 7f9c7adad700 1 -- 172.21.15.144:0/322049604 wait complete. 2023-11-28T11:51:46.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:46.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.137+0000 7f0624bbf700 1 Processor -- start 2023-11-28T11:51:46.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.137+0000 7f0624bbf700 1 -- start start 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f0624bbf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06200cd8c0 0x7f0620140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f0624bbf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062006f850 0x7f062006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f0624bbf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062013ff00 con 0x7f06200cdc90 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f0624bbf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06201404c0 con 0x7f06200cd8c0 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f0624bbf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062015aa80 con 0x7f062006f850 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f061e59c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06200cd8c0 0x7f0620140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f061e59c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06200cd8c0 0x7f0620140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35470/0 (socket says 172.21.15.144:35470) 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.138+0000 7f061e59c700 1 -- 172.21.15.144:0/3715042053 learned_addr learned my addr 172.21.15.144:0/3715042053 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061ed9d700 1 --2- 172.21.15.144:0/3715042053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062006f850 0x7f062006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061e59c700 1 -- 172.21.15.144:0/3715042053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062006f850 msgr2=0x7f062006fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:46.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061e59c700 1 --2- 172.21.15.144:0/3715042053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062006f850 0x7f062006fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061e59c700 1 -- 172.21.15.144:0/3715042053 >> v1:172.21.15.144:6789/0 conn(0x7f06200cdc90 legacy=0x7f06200b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061e59c700 1 -- 172.21.15.144:0/3715042053 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06200b3040 con 0x7f06200cd8c0 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061e59c700 1 --2- 172.21.15.144:0/3715042053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06200cd8c0 0x7f0620140950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0614006fd0 tx=0x7f0614009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4bab6bbccc0c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061d59a700 1 -- 172.21.15.144:0/3715042053 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614005e30 con 0x7f06200cd8c0 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061d59a700 1 -- 172.21.15.144:0/3715042053 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614004030 con 0x7f06200cd8c0 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.139+0000 7f061d59a700 1 -- 172.21.15.144:0/3715042053 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140043e0 con 0x7f06200cd8c0 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 -- 172.21.15.144:0/3715042053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06200cd8c0 msgr2=0x7f0620140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 --2- 172.21.15.144:0/3715042053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06200cd8c0 0x7f0620140950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0614006fd0 tx=0x7f0614009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 -- 172.21.15.144:0/3715042053 shutdown_connections 2023-11-28T11:51:46.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 --2- 172.21.15.144:0/3715042053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f062006f850 0x7f062006fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 --2- 172.21.15.144:0/3715042053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06200cd8c0 0x7f0620140950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 -- 172.21.15.144:0/3715042053 >> 172.21.15.144:0/3715042053 conn(0x7f062006aed0 msgr2=0x7f062006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:46.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 -- 172.21.15.144:0/3715042053 shutdown_connections 2023-11-28T11:51:46.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.140+0000 7f0624bbf700 1 -- 172.21.15.144:0/3715042053 wait complete. 2023-11-28T11:51:46.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.141+0000 7f0624bbf700 1 Processor -- start 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.141+0000 7f0624bbf700 1 -- start start 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f0624bbf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 0x7f0620143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f0624bbf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06200cd8c0 0x7f06200b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f0624bbf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06200cdc90 0x7f06200c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f0624bbf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200b4c50 con 0x7f062006f850 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f0624bbf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200b4f40 con 0x7f06200cd8c0 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f0624bbf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06200b5220 con 0x7f06200cdc90 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 0x7f0620143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 0x7f0620143c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35478/0 (socket says 172.21.15.144:35478) 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 -- 172.21.15.144:0/2867916206 learned_addr learned my addr 172.21.15.144:0/2867916206 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:46.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061ed9d700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06200cdc90 0x7f06200c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 -- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06200cdc90 msgr2=0x7f06200c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061dd9b700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06200cd8c0 0x7f06200b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06200cdc90 0x7f06200c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 -- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06200cd8c0 msgr2=0x7f06200b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06200cd8c0 0x7f06200b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 -- 172.21.15.144:0/2867916206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009500 con 0x7f062006f850 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061dd9b700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06200cd8c0 0x7f06200b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f061e59c700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 0x7f0620143c40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0614009c10 tx=0x7f0614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.142+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614011620 con 0x7f062006f850 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.143+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614011780 con 0x7f062006f850 2023-11-28T11:51:46.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.143+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06200b35f0 con 0x7f062006f850 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.143+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614011b30 con 0x7f062006f850 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.143+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06200b3310 con 0x7f062006f850 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.143+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0620140150 con 0x7f062006f850 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.144+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f061402c070 con 0x7f062006f850 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.144+0000 7f060effd700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06000682f0 0x7f060006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.144+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0614086f40 con 0x7f062006f850 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.144+0000 7f061dd9b700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06000682f0 0x7f060006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.145+0000 7f061dd9b700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06000682f0 0x7f060006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f06200b57e0 tx=0x7f0610009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.148+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06140b85e0 con 0x7f062006f850 2023-11-28T11:51:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.329+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0620068690 con 0x7f062006f850 2023-11-28T11:51:46.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.330+0000 7f060effd700 1 -- 172.21.15.144:0/2867916206 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f061402a0a0 con 0x7f062006f850 2023-11-28T11:51:46.332 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06000682f0 msgr2=0x7f060006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06000682f0 0x7f060006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f06200b57e0 tx=0x7f0610009040 comp rx=0 tx=0).stop 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 msgr2=0x7f0620143c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 0x7f0620143c40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0614009c10 tx=0x7f0614004750 comp rx=0 tx=0).stop 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 shutdown_connections 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062006f850 0x7f0620143c40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06000682f0 0x7f060006a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06200cdc90 0x7f06200c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 --2- 172.21.15.144:0/2867916206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f06200cd8c0 0x7f06200b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 >> 172.21.15.144:0/2867916206 conn(0x7f062006aed0 msgr2=0x7f062013f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:46.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 shutdown_connections 2023-11-28T11:51:46.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.332+0000 7f0624bbf700 1 -- 172.21.15.144:0/2867916206 wait complete. 2023-11-28T11:51:46.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:46.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.466+0000 7f0010595700 1 Processor -- start 2023-11-28T11:51:46.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.466+0000 7f0010595700 1 -- start start 2023-11-28T11:51:46.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f0010595700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0008071040 0x7f0008071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f0010595700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0008071950 con 0x7f0008160540 2023-11-28T11:51:46.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f0010595700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000815d060 con 0x7f000806dee0 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f0010595700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000815d340 con 0x7f0008071040 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f000eb32700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0008160540 0x7f0008160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52200/0 (socket says 172.21.15.144:52200) 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f000eb32700 1 -- 172.21.15.144:0/3265305936 learned_addr learned my addr 172.21.15.144:0/3265305936 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f000e331700 1 --2- 172.21.15.144:0/3265305936 >> v2:172.21.15.177:3300/0 conn(0x7f0008071040 0x7f0008071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805917110 0 0) 0x7f0008071950 con 0x7f0008160540 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.467+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff40035b0 con 0x7f0008160540 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45632765 0 0) 0x7f000815d060 con 0x7f000806dee0 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0008071950 con 0x7f000806dee0 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000e331700 1 -- 172.21.15.144:0/3265305936 >> v1:172.21.15.177:6789/0 conn(0x7f000806dee0 legacy=0x7f000806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542885907 0 0) 0x7efff40035b0 con 0x7f0008160540 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000e331700 1 -- 172.21.15.144:0/3265305936 >> v1:172.21.15.144:6789/0 conn(0x7f0008160540 legacy=0x7f0008160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000e331700 1 -- 172.21.15.144:0/3265305936 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000815d620 con 0x7f0008071040 2023-11-28T11:51:46.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000e331700 1 --2- 172.21.15.144:0/3265305936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0008071040 0x7f0008071410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efff8009840 tx=0x7efff8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c36184bb62ff76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00000019f0 con 0x7f0008071040 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff8004030 con 0x7f0008071040 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f000d32f700 1 -- 172.21.15.144:0/3265305936 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff80043c0 con 0x7f0008071040 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f0010595700 1 -- 172.21.15.144:0/3265305936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0008071040 msgr2=0x7f0008071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.468+0000 7f0010595700 1 --2- 172.21.15.144:0/3265305936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0008071040 0x7f0008071410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efff8009840 tx=0x7efff8009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.469+0000 7f0010595700 1 -- 172.21.15.144:0/3265305936 shutdown_connections 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.469+0000 7f0010595700 1 --2- 172.21.15.144:0/3265305936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0008071040 0x7f0008071410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.469+0000 7f0010595700 1 -- 172.21.15.144:0/3265305936 >> 172.21.15.144:0/3265305936 conn(0x7f00080d0000 msgr2=0x7f00080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.469+0000 7f0010595700 1 -- 172.21.15.144:0/3265305936 shutdown_connections 2023-11-28T11:51:46.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.469+0000 7f0010595700 1 -- 172.21.15.144:0/3265305936 wait complete. 2023-11-28T11:51:46.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.470+0000 7f0010595700 1 Processor -- start 2023-11-28T11:51:46.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.470+0000 7f0010595700 1 -- start start 2023-11-28T11:51:46.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f000806dee0 0x7f00081f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0008071040 0x7f0008150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 0x7f00080c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00080c5b30 con 0x7f0008160540 2023-11-28T11:51:46.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00080c68e0 con 0x7f000806dee0 2023-11-28T11:51:46.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00080c5d60 con 0x7f0008071040 2023-11-28T11:51:46.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 0x7f00080c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 0x7f00080c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35494/0 (socket says 172.21.15.144:35494) 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 -- 172.21.15.144:0/3195271346 learned_addr learned my addr 172.21.15.144:0/3195271346 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000e331700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f000806dee0 0x7f00081f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000db30700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0008071040 0x7f0008150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 -- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0008071040 msgr2=0x7f0008150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0008071040 0x7f0008150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 -- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f000806dee0 msgr2=0x7f00081f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f000806dee0 0x7f00081f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 -- 172.21.15.144:0/3195271346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff80094a0 con 0x7f0008160540 2023-11-28T11:51:46.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000db30700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0008071040 0x7f0008150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f000eb32700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 0x7f00080c55f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f000000cbc0 tx=0x7f000000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000000abb0 con 0x7f0008160540 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000000ad30 con 0x7f0008160540 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000815ccd0 con 0x7f0008160540 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000000aea0 con 0x7f0008160540 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.471+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000815d900 con 0x7f0008160540 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.472+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000806a270 con 0x7f0008160540 2023-11-28T11:51:46.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.473+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0000010070 con 0x7f0008160540 2023-11-28T11:51:46.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.473+0000 7efffeffd700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effec0682f0 0x7effec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.473+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0000085710 con 0x7f0008160540 2023-11-28T11:51:46.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.473+0000 7f000e331700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effec0682f0 0x7effec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.474+0000 7f000e331700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effec0682f0 0x7effec06a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efff80090b0 tx=0x7efff8013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.477+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0000051f10 con 0x7f0008160540 2023-11-28T11:51:46.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.661+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00080720a0 con 0x7f0008160540 2023-11-28T11:51:46.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.662+0000 7efffeffd700 1 -- 172.21.15.144:0/3195271346 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0000055530 con 0x7f0008160540 2023-11-28T11:51:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effec0682f0 msgr2=0x7effec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effec0682f0 0x7effec06a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efff80090b0 tx=0x7efff8013040 comp rx=0 tx=0).stop 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 msgr2=0x7f00080c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 0x7f00080c55f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f000000cbc0 tx=0x7f000000cf80 comp rx=0 tx=0).stop 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 shutdown_connections 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0008160540 0x7f00080c55f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effec0682f0 0x7effec06a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f000806dee0 0x7f00081f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 --2- 172.21.15.144:0/3195271346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0008071040 0x7f0008150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 >> 172.21.15.144:0/3195271346 conn(0x7f00080d0000 msgr2=0x7f00080cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 shutdown_connections 2023-11-28T11:51:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.664+0000 7f0010595700 1 -- 172.21.15.144:0/3195271346 wait complete. 2023-11-28T11:51:46.678 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:46.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.801+0000 7fdb5edf1700 1 Processor -- start 2023-11-28T11:51:46.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb5edf1700 1 -- start start 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb5edf1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58071040 0x7fdb58071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb5edf1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb5806dee0 0x7fdb5806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb5edf1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb58071950 con 0x7fdb5806cdc0 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb5edf1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb58161550 con 0x7fdb5806dee0 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb5edf1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815d060 con 0x7fdb58071040 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb57fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb5806dee0 0x7fdb5806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb57fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb5806dee0 0x7fdb5806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35498/0 (socket says 172.21.15.144:35498) 2023-11-28T11:51:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.802+0000 7fdb57fff700 1 -- 172.21.15.144:0/3757972753 learned_addr learned my addr 172.21.15.144:0/3757972753 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb5cb8d700 1 --2- 172.21.15.144:0/3757972753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58071040 0x7fdb58071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb57fff700 1 -- 172.21.15.144:0/3757972753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58071040 msgr2=0x7fdb58071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb57fff700 1 --2- 172.21.15.144:0/3757972753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58071040 0x7fdb58071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb57fff700 1 -- 172.21.15.144:0/3757972753 >> v1:172.21.15.177:6789/0 conn(0x7fdb5806cdc0 legacy=0x7fdb5806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb57fff700 1 -- 172.21.15.144:0/3757972753 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5815d340 con 0x7fdb5806dee0 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb57fff700 1 --2- 172.21.15.144:0/3757972753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5806dee0 0x7fdb5806e350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb440098f0 tx=0x7fdb44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2891552b98988dfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb577fe700 1 -- 172.21.15.144:0/3757972753 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400b8a0 con 0x7fdb5806dee0 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb577fe700 1 -- 172.21.15.144:0/3757972753 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4400ba00 con 0x7fdb5806dee0 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb577fe700 1 -- 172.21.15.144:0/3757972753 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400bd90 con 0x7fdb5806dee0 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.803+0000 7fdb5edf1700 1 -- 172.21.15.144:0/3757972753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5806dee0 msgr2=0x7fdb5806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/3757972753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5806dee0 0x7fdb5806e350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb440098f0 tx=0x7fdb44009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 -- 172.21.15.144:0/3757972753 shutdown_connections 2023-11-28T11:51:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/3757972753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb58071040 0x7fdb58071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/3757972753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb5806dee0 0x7fdb5806e350 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 -- 172.21.15.144:0/3757972753 >> 172.21.15.144:0/3757972753 conn(0x7fdb580d0000 msgr2=0x7fdb580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 -- 172.21.15.144:0/3757972753 shutdown_connections 2023-11-28T11:51:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.804+0000 7fdb5edf1700 1 -- 172.21.15.144:0/3757972753 wait complete. 2023-11-28T11:51:46.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.805+0000 7fdb5edf1700 1 Processor -- start 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.805+0000 7fdb5edf1700 1 -- start start 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806cdc0 0x7fdb5815f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb5806dee0 0x7fdb58149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 0x7fdb5814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580c3d60 con 0x7fdb58071040 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581f5f50 con 0x7fdb5806cdc0 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581f61a0 con 0x7fdb5806dee0 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 0x7fdb5814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 0x7fdb5814a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35508/0 (socket says 172.21.15.144:35508) 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 -- 172.21.15.144:0/2765341443 learned_addr learned my addr 172.21.15.144:0/2765341443 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb57fff700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb5806dee0 0x7fdb58149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5cb8d700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806cdc0 0x7fdb5815f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 -- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb5806dee0 msgr2=0x7fdb58149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb5806dee0 0x7fdb58149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 -- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806cdc0 msgr2=0x7fdb5815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806cdc0 0x7fdb5815f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 -- 172.21.15.144:0/2765341443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009550 con 0x7fdb58071040 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5cb8d700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806cdc0 0x7fdb5815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb57fff700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb5806dee0 0x7fdb58149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5d38e700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 0x7fdb5814a470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000cb50 tx=0x7fdb5000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50003c00 con 0x7fdb58071040 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50003d60 con 0x7fdb58071040 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5815d900 con 0x7fdb58071040 2023-11-28T11:51:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.806+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5000ab70 con 0x7fdb58071040 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.807+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5815d620 con 0x7fdb58071040 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.807+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5806e890 con 0x7fdb58071040 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.808+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb50010050 con 0x7fdb58071040 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.808+0000 7fdb557fa700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.808+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb50085b70 con 0x7fdb58071040 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.808+0000 7fdb5cb8d700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:46.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.809+0000 7fdb5cb8d700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdb581576a0 tx=0x7fdb4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:46.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.812+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb500523d0 con 0x7fdb58071040 2023-11-28T11:51:46.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.995+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb580af740 con 0x7fdb58071040 2023-11-28T11:51:46.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.995+0000 7fdb557fa700 1 -- 172.21.15.144:0/2765341443 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb500559f0 con 0x7fdb58071040 2023-11-28T11:51:46.997 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:46.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb3c0682f0 msgr2=0x7fdb3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdb581576a0 tx=0x7fdb4c008040 comp rx=0 tx=0).stop 2023-11-28T11:51:46.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 msgr2=0x7fdb5814a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:46.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 0x7fdb5814a470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000cb50 tx=0x7fdb5000cf10 comp rx=0 tx=0).stop 2023-11-28T11:51:46.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 shutdown_connections 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58071040 0x7fdb5814a470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb5806cdc0 0x7fdb5815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 --2- 172.21.15.144:0/2765341443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb5806dee0 0x7fdb58149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 >> 172.21.15.144:0/2765341443 conn(0x7fdb580d0000 msgr2=0x7fdb5806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 shutdown_connections 2023-11-28T11:51:47.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:46.997+0000 7fdb5edf1700 1 -- 172.21.15.144:0/2765341443 wait complete. 2023-11-28T11:51:47.012 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:51:47.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.131+0000 7f7ec2bb6700 1 Processor -- start 2023-11-28T11:51:47.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.131+0000 7f7ec2bb6700 1 -- start start 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec2bb6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 0x7f7ebc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec2bb6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc06b420 con 0x7f7ebc0b2930 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec2bb6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc06b700 con 0x7f7ebc0ce040 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec2bb6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06ff60 con 0x7f7ebc0d0a00 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec1153700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7ebc0b2930 0x7f7ebc06c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52226/0 (socket says 172.21.15.144:52226) 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec1153700 1 -- 172.21.15.144:0/1093580018 learned_addr learned my addr 172.21.15.144:0/1093580018 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:47.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebbfff700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 0x7f7ebc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529666986 0 0) 0x7f7ebc06b420 con 0x7f7ebc0b2930 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac0035b0 con 0x7f7ebc0b2930 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984802622 0 0) 0x7f7ebc06b700 con 0x7f7ebc0ce040 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ebc06b420 con 0x7f7ebc0ce040 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016578849 0 0) 0x7f7eac0035b0 con 0x7f7ebc0b2930 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 msgr2=0x7f7ebc15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 0x7f7ebc15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 >> v1:172.21.15.177:6789/0 conn(0x7f7ebc0ce040 legacy=0x7f7ebc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc06b9e0 con 0x7f7ebc0b2930 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f7eb40019f0 con 0x7f7ebc0b2930 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 >> v1:172.21.15.144:6789/0 conn(0x7f7ebc0b2930 legacy=0x7f7ebc06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7eac003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eac004850 0x7f7eac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 0x7f7eac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06ff60 con 0x7f7eac003bc0 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac00bac0 con 0x7f7eac004850 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac00bc30 con 0x7f7ebc0d0a00 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.132+0000 7f7ec0952700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7eac003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ebbfff700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eac004850 0x7f7eac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec1153700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 0x7f7eac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec0952700 1 -- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 msgr2=0x7f7eac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec0952700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0d0a00 0x7f7eac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec0952700 1 -- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eac004850 msgr2=0x7f7eac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:47.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec0952700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7eac004850 0x7f7eac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec0952700 1 -- 172.21.15.144:0/1093580018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0035b0 con 0x7f7eac003bc0 2023-11-28T11:51:47.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ec0952700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7eac003fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00019a0 tx=0x7f7eac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a8885bd47f3afc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:47.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0001dd0 con 0x7f7eac003bc0 2023-11-28T11:51:47.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb00052d0 con 0x7f7eac003bc0 2023-11-28T11:51:47.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.133+0000 7f7ebb7fe700 1 -- 172.21.15.144:0/1093580018 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0001460 con 0x7f7eac003bc0 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/1093580018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 msgr2=0x7f7eac003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/1093580018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7eac003fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00019a0 tx=0x7f7eac005880 comp rx=0 tx=0).stop 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec1153700 1 -- 172.21.15.144:0/1093580018 reap_dead start 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/1093580018 shutdown_connections 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/1093580018 >> 172.21.15.144:0/1093580018 conn(0x7f7ebc06aed0 msgr2=0x7f7ebc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/1093580018 shutdown_connections 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.134+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/1093580018 wait complete. 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.135+0000 7f7ec2bb6700 1 Processor -- start 2023-11-28T11:51:47.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.135+0000 7f7ec2bb6700 1 -- start start 2023-11-28T11:51:47.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7ebc1523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ebc0ccad0 0x7f7ebc0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0c8990 0x7f7ebc0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0cd480 con 0x7f7eac003bc0 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc152910 con 0x7f7ebc0ccad0 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0ca760 con 0x7f7ebc0c8990 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7ebc1523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7ebc1523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35522/0 (socket says 172.21.15.144:35522) 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 -- 172.21.15.144:0/4042437964 learned_addr learned my addr 172.21.15.144:0/4042437964 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec1153700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0c8990 0x7f7ebc0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ebbfff700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ebc0ccad0 0x7f7ebc0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 -- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0c8990 msgr2=0x7f7ebc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0c8990 0x7f7ebc0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 -- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ebc0ccad0 msgr2=0x7f7ebc0ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ebc0ccad0 0x7f7ebc0ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 -- 172.21.15.144:0/4042437964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc06bfa0 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec1153700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0c8990 0x7f7ebc0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ebbfff700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ebc0ccad0 0x7f7ebc0ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec0952700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7ebc1523d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0000c00 tx=0x7f7eb0001320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0001dd0 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb0001400 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0006e10 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc06bcc0 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.136+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc15df10 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.137+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc068690 con 0x7f7eac003bc0 2023-11-28T11:51:47.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.138+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7eb002d070 con 0x7f7eac003bc0 2023-11-28T11:51:47.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.138+0000 7f7eb97fa700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ea0068320 0x7f7ea006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.138+0000 7f7ebbfff700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ea0068320 0x7f7ea006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.138+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7eb00865a0 con 0x7f7eac003bc0 2023-11-28T11:51:47.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.139+0000 7f7ebbfff700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ea0068320 0x7f7ea006a7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc15f710 tx=0x7f7ea8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:47.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.143+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7eb0052e00 con 0x7f7eac003bc0 2023-11-28T11:51:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.338+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ebc15ea10 con 0x7f7eac003bc0 2023-11-28T11:51:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.338+0000 7f7eb97fa700 1 -- 172.21.15.144:0/4042437964 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7eb0056420 con 0x7f7eac003bc0 2023-11-28T11:51:47.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ea0068320 msgr2=0x7f7ea006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ea0068320 0x7f7ea006a7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc15f710 tx=0x7f7ea8008040 comp rx=0 tx=0).stop 2023-11-28T11:51:47.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 msgr2=0x7f7ebc1523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7ebc1523d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0000c00 tx=0x7f7eb0001320 comp rx=0 tx=0).stop 2023-11-28T11:51:47.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 shutdown_connections 2023-11-28T11:51:47.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7eac003bc0 0x7f7ebc1523d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7ea0068320 0x7f7ea006a7d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ebc0c8990 0x7f7ebc0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 --2- 172.21.15.144:0/4042437964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ebc0ccad0 0x7f7ebc0ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 >> 172.21.15.144:0/4042437964 conn(0x7f7ebc06aed0 msgr2=0x7f7ebc15ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 shutdown_connections 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.340+0000 7f7ec2bb6700 1 -- 172.21.15.144:0/4042437964 wait complete. 2023-11-28T11:51:47.343 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:51:47.354 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:51:47.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:47.354 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T11:51:47.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:51:47.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:47.394 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:51:47.456 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:47.456 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:47.478 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:47.479 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:47.540 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:47.541 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:47.601 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:47.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 Processor -- start 2023-11-28T11:51:47.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 -- start start 2023-11-28T11:51:47.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd51006ba90 0x7fd5100cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd5100ccc40 0x7fd5100cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100cf5e0 con 0x7fd51015ca50 2023-11-28T11:51:47.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100cd470 con 0x7fd51006ba90 2023-11-28T11:51:47.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.719+0000 7fd515432700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100cecb0 con 0x7fd5100ccc40 2023-11-28T11:51:47.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd51006ba90 0x7fd5100cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd51006ba90 0x7fd5100cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35532/0 (socket says 172.21.15.144:35532) 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 -- 172.21.15.144:0/3634263763 learned_addr learned my addr 172.21.15.144:0/3634263763 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50f7fe700 1 --2- 172.21.15.144:0/3634263763 >> v2:172.21.15.177:3300/0 conn(0x7fd5100ccc40 0x7fd5100cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 -- 172.21.15.144:0/3634263763 >> v2:172.21.15.177:3300/0 conn(0x7fd5100ccc40 msgr2=0x7fd5100cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 --2- 172.21.15.144:0/3634263763 >> v2:172.21.15.177:3300/0 conn(0x7fd5100ccc40 0x7fd5100cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 -- 172.21.15.144:0/3634263763 >> v1:172.21.15.144:6789/0 conn(0x7fd51015ca50 legacy=0x7fd51006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.730 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 -- 172.21.15.144:0/3634263763 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100cef90 con 0x7fd51006ba90 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50f7fe700 1 --2- 172.21.15.144:0/3634263763 >> v2:172.21.15.177:3300/0 conn(0x7fd5100ccc40 0x7fd5100cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50e7fc700 1 --2- 172.21.15.144:0/3634263763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51006ba90 0x7fd5100cc700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd5100c7fc0 tx=0x7fd504009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9525cfcb7682a03 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50dffb700 1 -- 172.21.15.144:0/3634263763 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400b870 con 0x7fd51006ba90 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50dffb700 1 -- 172.21.15.144:0/3634263763 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50400b9d0 con 0x7fd51006ba90 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.720+0000 7fd50dffb700 1 -- 172.21.15.144:0/3634263763 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400bd60 con 0x7fd51006ba90 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 -- 172.21.15.144:0/3634263763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51006ba90 msgr2=0x7fd5100cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 --2- 172.21.15.144:0/3634263763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51006ba90 0x7fd5100cc700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd5100c7fc0 tx=0x7fd504009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 -- 172.21.15.144:0/3634263763 shutdown_connections 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 --2- 172.21.15.144:0/3634263763 >> v2:172.21.15.177:3300/0 conn(0x7fd5100ccc40 0x7fd5100cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 --2- 172.21.15.144:0/3634263763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51006ba90 0x7fd5100cc700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 -- 172.21.15.144:0/3634263763 >> 172.21.15.144:0/3634263763 conn(0x7fd51009a230 msgr2=0x7fd51008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 -- 172.21.15.144:0/3634263763 shutdown_connections 2023-11-28T11:51:47.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.721+0000 7fd515432700 1 -- 172.21.15.144:0/3634263763 wait complete. 2023-11-28T11:51:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.722+0000 7fd515432700 1 Processor -- start 2023-11-28T11:51:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.722+0000 7fd515432700 1 -- start start 2023-11-28T11:51:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd515432700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51006ba90 0x7fd510158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd515432700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5100ccc40 0x7fd510150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd515432700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 0x7fd5100b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd515432700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100c71d0 con 0x7fd51015ca50 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd515432700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510157730 con 0x7fd5100ccc40 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd515432700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd510157f90 con 0x7fd51006ba90 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.723+0000 7fd50f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 0x7fd5100b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 0x7fd5100b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35540/0 (socket says 172.21.15.144:35540) 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 -- 172.21.15.144:0/2827232909 learned_addr learned my addr 172.21.15.144:0/2827232909 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50effd700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51006ba90 0x7fd510158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50e7fc700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5100ccc40 0x7fd510150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 -- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51006ba90 msgr2=0x7fd510158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51006ba90 0x7fd510158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 -- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5100ccc40 msgr2=0x7fd510150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5100ccc40 0x7fd510150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 -- 172.21.15.144:0/2827232909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504009550 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd50f7fe700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 0x7fd5100b2580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd50800cef0 tx=0x7fd50800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50800adf0 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508015440 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5100cd060 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080157f0 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.724+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5100cf270 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.725+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5100cd1f0 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.725+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd50801c030 con 0x7fd51015ca50 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.726+0000 7fd4f77fe700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd4fc0682f0 0x7fd4fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.726+0000 7fd50effd700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd4fc0682f0 0x7fd4fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:47.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.726+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd508086080 con 0x7fd51015ca50 2023-11-28T11:51:47.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.727+0000 7fd50effd700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd4fc0682f0 0x7fd4fc06a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd5101f59f0 tx=0x7fd500009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:47.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.730+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd5080528e0 con 0x7fd51015ca50 2023-11-28T11:51:47.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.912+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5100c6f50 con 0x7fd51015ca50 2023-11-28T11:51:47.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.913+0000 7fd4f77fe700 1 -- 172.21.15.144:0/2827232909 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7fd508055f00 con 0x7fd51015ca50 2023-11-28T11:51:47.915 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:47.916 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd4fc0682f0 msgr2=0x7fd4fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd4fc0682f0 0x7fd4fc06a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd5101f59f0 tx=0x7fd500009040 comp rx=0 tx=0).stop 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 msgr2=0x7fd5100b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 0x7fd5100b2580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd50800cef0 tx=0x7fd50800a340 comp rx=0 tx=0).stop 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 shutdown_connections 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51015ca50 0x7fd5100b2580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd4fc0682f0 0x7fd4fc06a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51006ba90 0x7fd510158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 --2- 172.21.15.144:0/2827232909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5100ccc40 0x7fd510150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:47.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 >> 172.21.15.144:0/2827232909 conn(0x7fd51009a230 msgr2=0x7fd5100afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:47.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 shutdown_connections 2023-11-28T11:51:47.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:47.916+0000 7fd515432700 1 -- 172.21.15.144:0/2827232909 wait complete. 2023-11-28T11:51:47.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:47.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:48.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.051+0000 7f046cc6d700 1 Processor -- start 2023-11-28T11:51:48.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.051+0000 7f046cc6d700 1 -- start start 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046cc6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0468154e70 0x7f04680cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046cc6d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f04680cf360 0x7f0468071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046cc6d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04680719e0 con 0x7f0468071580 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046cc6d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0468155240 con 0x7f04680cf360 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046cc6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046815b100 con 0x7f0468154e70 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0468154e70 0x7f04680cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f0465d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f04680cf360 0x7f0468071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0468154e70 0x7f04680cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47626/0 (socket says 172.21.15.144:47626) 2023-11-28T11:51:48.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f046659c700 1 -- 172.21.15.144:0/724121771 learned_addr learned my addr 172.21.15.144:0/724121771 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.052+0000 7f0466d9d700 1 --1- 172.21.15.144:0/724121771 >> v1:172.21.15.177:6789/0 conn(0x7f0468071580 0x7f046815e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:52748/0 (socket says 172.21.15.144:52748) 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f0465d9b700 1 -- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0468154e70 msgr2=0x7f04680cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f0465d9b700 1 --2- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0468154e70 0x7f04680cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f0465d9b700 1 -- 172.21.15.144:0/724121771 >> v1:172.21.15.177:6789/0 conn(0x7f0468071580 legacy=0x7f046815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f0465d9b700 1 -- 172.21.15.144:0/724121771 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046815b3e0 con 0x7f04680cf360 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f0465d9b700 1 --2- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04680cf360 0x7f0468071040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0458009840 tx=0x7f0458009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb9a4bcfaf6827c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f046559a700 1 -- 172.21.15.144:0/724121771 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458011830 con 0x7f04680cf360 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f046559a700 1 -- 172.21.15.144:0/724121771 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0458011990 con 0x7f04680cf360 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.053+0000 7f046559a700 1 -- 172.21.15.144:0/724121771 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458011d20 con 0x7f04680cf360 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 -- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04680cf360 msgr2=0x7f0468071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 --2- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04680cf360 0x7f0468071040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0458009840 tx=0x7f0458009b50 comp rx=0 tx=0).stop 2023-11-28T11:51:48.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 -- 172.21.15.144:0/724121771 shutdown_connections 2023-11-28T11:51:48.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 --2- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0468154e70 0x7f04680cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 --2- 172.21.15.144:0/724121771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04680cf360 0x7f0468071040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 -- 172.21.15.144:0/724121771 >> 172.21.15.144:0/724121771 conn(0x7f04680d0000 msgr2=0x7f04680d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:48.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 -- 172.21.15.144:0/724121771 shutdown_connections 2023-11-28T11:51:48.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.054+0000 7f046cc6d700 1 -- 172.21.15.144:0/724121771 wait complete. 2023-11-28T11:51:48.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.055+0000 7f046cc6d700 1 Processor -- start 2023-11-28T11:51:48.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.055+0000 7f046cc6d700 1 -- start start 2023-11-28T11:51:48.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046cc6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 0x7f046814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046cc6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04680cf360 0x7f04680b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046cc6d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0468154e70 0x7f04680b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046cc6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04680cd810 con 0x7f0468071580 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046cc6d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04680cd250 con 0x7f0468154e70 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046cc6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04680cd530 con 0x7f04680cf360 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 0x7f046814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 0x7f046814c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35550/0 (socket says 172.21.15.144:35550) 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 -- 172.21.15.144:0/1135674934 learned_addr learned my addr 172.21.15.144:0/1135674934 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f0466d9d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0468154e70 0x7f04680b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f0465d9b700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04680cf360 0x7f04680b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 -- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04680cf360 msgr2=0x7f04680b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04680cf360 0x7f04680b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 -- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0468154e70 msgr2=0x7f04680b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0468154e70 0x7f04680b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 -- 172.21.15.144:0/1135674934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04580094a0 con 0x7f0468071580 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f0465d9b700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04680cf360 0x7f04680b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f046659c700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 0x7f046814c3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f045c00e8b0 tx=0x7f045c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.056+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c009600 con 0x7f0468071580 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.057+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046815b9a0 con 0x7f0468071580 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.057+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045c00eeb0 con 0x7f0468071580 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.057+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c00ca70 con 0x7f0468071580 2023-11-28T11:51:48.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.057+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046815b6c0 con 0x7f0468071580 2023-11-28T11:51:48.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.058+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04680af520 con 0x7f0468071580 2023-11-28T11:51:48.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.058+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f045c019030 con 0x7f0468071580 2023-11-28T11:51:48.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.058+0000 7f0456ffd700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04480682f0 0x7f044806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.059+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f045c085b00 con 0x7f0468071580 2023-11-28T11:51:48.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.059+0000 7f0465d9b700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04480682f0 0x7f044806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.059+0000 7f0465d9b700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04480682f0 0x7f044806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f046809e000 tx=0x7f0458008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.063+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f045c0522e0 con 0x7f0468071580 2023-11-28T11:51:48.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.244+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f046806d5b0 con 0x7f0468071580 2023-11-28T11:51:48.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.244+0000 7f0456ffd700 1 -- 172.21.15.144:0/1135674934 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f045c055900 con 0x7f0468071580 2023-11-28T11:51:48.246 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:48.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04480682f0 msgr2=0x7f044806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04480682f0 0x7f044806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f046809e000 tx=0x7f0458008030 comp rx=0 tx=0).stop 2023-11-28T11:51:48.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 msgr2=0x7f046814c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 0x7f046814c3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f045c00e8b0 tx=0x7f045c00ec70 comp rx=0 tx=0).stop 2023-11-28T11:51:48.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 shutdown_connections 2023-11-28T11:51:48.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0468071580 0x7f046814c3f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04480682f0 0x7f044806a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0468154e70 0x7f04680b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 --2- 172.21.15.144:0/1135674934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04680cf360 0x7f04680b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 >> 172.21.15.144:0/1135674934 conn(0x7f04680d0000 msgr2=0x7f046806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:48.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 shutdown_connections 2023-11-28T11:51:48.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.246+0000 7f046cc6d700 1 -- 172.21.15.144:0/1135674934 wait complete. 2023-11-28T11:51:48.262 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:48.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.383+0000 7fd5cb91e700 1 Processor -- start 2023-11-28T11:51:48.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5cb91e700 1 -- start start 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5cb91e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd5c40cd8c0 0x7fd5c4140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5cb91e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c406f850 0x7fd5c406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5cb91e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c413ff00 con 0x7fd5c40cdc90 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5cb91e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c41404c0 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5cb91e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c415aa80 con 0x7fd5c406f850 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5c8eb9700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd5c40cdc90 0x7fd5c40b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52270/0 (socket says 172.21.15.144:52270) 2023-11-28T11:51:48.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.384+0000 7fd5c8eb9700 1 -- 172.21.15.144:0/1931349734 learned_addr learned my addr 172.21.15.144:0/1931349734 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c9ebb700 1 --2- 172.21.15.144:0/1931349734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c406f850 0x7fd5c406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c96ba700 1 --2- 172.21.15.144:0/1931349734 >> v2:172.21.15.177:3300/0 conn(0x7fd5c40cd8c0 0x7fd5c4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5bbfff700 1 -- 172.21.15.144:0/1931349734 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189576817 0 0) 0x7fd5c413ff00 con 0x7fd5c40cdc90 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5bbfff700 1 -- 172.21.15.144:0/1931349734 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b00035b0 con 0x7fd5c40cdc90 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c9ebb700 1 -- 172.21.15.144:0/1931349734 >> v2:172.21.15.177:3300/0 conn(0x7fd5c40cd8c0 msgr2=0x7fd5c4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c9ebb700 1 --2- 172.21.15.144:0/1931349734 >> v2:172.21.15.177:3300/0 conn(0x7fd5c40cd8c0 0x7fd5c4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c9ebb700 1 -- 172.21.15.144:0/1931349734 >> v1:172.21.15.144:6789/0 conn(0x7fd5c40cdc90 legacy=0x7fd5c40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c9ebb700 1 -- 172.21.15.144:0/1931349734 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c40b3040 con 0x7fd5c406f850 2023-11-28T11:51:48.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5c9ebb700 1 --2- 172.21.15.144:0/1931349734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c406f850 0x7fd5c406fcc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009d90 tx=0x7fd5bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a07e2e60f9b08f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5bbfff700 1 -- 172.21.15.144:0/1931349734 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00e070 con 0x7fd5c406f850 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5bbfff700 1 -- 172.21.15.144:0/1931349734 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc00bb20 con 0x7fd5c406f850 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.385+0000 7fd5bbfff700 1 -- 172.21.15.144:0/1931349734 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc012430 con 0x7fd5c406f850 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1931349734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c406f850 msgr2=0x7fd5c406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1931349734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c406f850 0x7fd5c406fcc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009d90 tx=0x7fd5bc00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1931349734 shutdown_connections 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1931349734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c406f850 0x7fd5c406fcc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1931349734 >> v2:172.21.15.177:3300/0 conn(0x7fd5c40cd8c0 0x7fd5c4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1931349734 >> 172.21.15.144:0/1931349734 conn(0x7fd5c406aed0 msgr2=0x7fd5c406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:48.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1931349734 shutdown_connections 2023-11-28T11:51:48.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.386+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1931349734 wait complete. 2023-11-28T11:51:48.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.387+0000 7fd5cb91e700 1 Processor -- start 2023-11-28T11:51:48.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.387+0000 7fd5cb91e700 1 -- start start 2023-11-28T11:51:48.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5cb91e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c406f850 0x7fd5c41f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5cb91e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 0x7fd5c41f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5cb91e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c40cdc90 0x7fd5c413ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5cb91e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c41f0f00 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5cb91e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c41f3230 con 0x7fd5c406f850 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5cb91e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4141190 con 0x7fd5c40cdc90 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 0x7fd5c41f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 0x7fd5c41f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35564/0 (socket says 172.21.15.144:35564) 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 -- 172.21.15.144:0/1591552669 learned_addr learned my addr 172.21.15.144:0/1591552669 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c9ebb700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c40cdc90 0x7fd5c413ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c96ba700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c406f850 0x7fd5c41f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 -- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c40cdc90 msgr2=0x7fd5c413ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c40cdc90 0x7fd5c413ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 -- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c406f850 msgr2=0x7fd5c41f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c406f850 0x7fd5c41f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 -- 172.21.15.144:0/1591552669 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc009a70 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c96ba700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c406f850 0x7fd5c41f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.388+0000 7fd5c8eb9700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 0x7fd5c41f2cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005910 tx=0x7fd5c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.389+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0005d70 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.389+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c00070d0 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.389+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c40b35f0 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.389+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0007440 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.389+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c40b3310 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.390+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c4140150 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.390+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd5c0019030 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.390+0000 7fd5b9ffb700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5a80682f0 0x7fd5a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.390+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5c0084ec0 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.390+0000 7fd5c96ba700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5a80682f0 0x7fd5a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.391+0000 7fd5c96ba700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5a80682f0 0x7fd5a806a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009c10 tx=0x7fd5b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.395+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd5c004b5e0 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.577+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5c4068690 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.578+0000 7fd5b9ffb700 1 -- 172.21.15.144:0/1591552669 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5c00170a0 con 0x7fd5c40cd8c0 2023-11-28T11:51:48.579 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5a80682f0 msgr2=0x7fd5a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5a80682f0 0x7fd5a806a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009c10 tx=0x7fd5b4008040 comp rx=0 tx=0).stop 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 msgr2=0x7fd5c41f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 0x7fd5c41f2cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005910 tx=0x7fd5c000c860 comp rx=0 tx=0).stop 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 shutdown_connections 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5a80682f0 0x7fd5a806a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5c40cdc90 0x7fd5c413ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c406f850 0x7fd5c41f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 --2- 172.21.15.144:0/1591552669 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c40cd8c0 0x7fd5c41f2cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 >> 172.21.15.144:0/1591552669 conn(0x7fd5c406aed0 msgr2=0x7fd5c406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:48.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 shutdown_connections 2023-11-28T11:51:48.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.580+0000 7fd5cb91e700 1 -- 172.21.15.144:0/1591552669 wait complete. 2023-11-28T11:51:48.594 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:48.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.715+0000 7fea6afd4700 1 Processor -- start 2023-11-28T11:51:48.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.715+0000 7fea6afd4700 1 -- start start 2023-11-28T11:51:48.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea6afd4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea6406ba90 0x7fea640af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea6afd4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 0x7fea640ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea6afd4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea640cc5f0 con 0x7fea6415cef0 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea6afd4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6415d470 con 0x7fea6406ba90 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea6afd4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6415d750 con 0x7fea640cbdf0 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea63fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea6406ba90 0x7fea640af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea69571700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 0x7fea640ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea69571700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 0x7fea640ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47652/0 (socket says 172.21.15.144:47652) 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.716+0000 7fea69571700 1 -- 172.21.15.144:0/2656129004 learned_addr learned my addr 172.21.15.144:0/2656129004 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea69571700 1 -- 172.21.15.144:0/2656129004 >> v2:172.21.15.177:3300/0 conn(0x7fea6406ba90 msgr2=0x7fea640af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea69571700 1 --2- 172.21.15.144:0/2656129004 >> v2:172.21.15.177:3300/0 conn(0x7fea6406ba90 0x7fea640af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea69571700 1 -- 172.21.15.144:0/2656129004 >> v1:172.21.15.177:6789/0 conn(0x7fea6415cef0 legacy=0x7fea6406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea69571700 1 -- 172.21.15.144:0/2656129004 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6415da30 con 0x7fea640cbdf0 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea63fff700 1 --2- 172.21.15.144:0/2656129004 >> v2:172.21.15.177:3300/0 conn(0x7fea6406ba90 0x7fea640af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea69571700 1 --2- 172.21.15.144:0/2656129004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 0x7fea640ca650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009e50 tx=0x7fea5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d79659404136ed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea637fe700 1 -- 172.21.15.144:0/2656129004 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00e070 con 0x7fea640cbdf0 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea637fe700 1 -- 172.21.15.144:0/2656129004 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c00b910 con 0x7fea640cbdf0 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.717+0000 7fea637fe700 1 -- 172.21.15.144:0/2656129004 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00bcc0 con 0x7fea640cbdf0 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 -- 172.21.15.144:0/2656129004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 msgr2=0x7fea640ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 --2- 172.21.15.144:0/2656129004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 0x7fea640ca650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009e50 tx=0x7fea5c00b330 comp rx=0 tx=0).stop 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 -- 172.21.15.144:0/2656129004 shutdown_connections 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 --2- 172.21.15.144:0/2656129004 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea640cbdf0 0x7fea640ca650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 --2- 172.21.15.144:0/2656129004 >> v2:172.21.15.177:3300/0 conn(0x7fea6406ba90 0x7fea640af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 -- 172.21.15.144:0/2656129004 >> 172.21.15.144:0/2656129004 conn(0x7fea640d0000 msgr2=0x7fea640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:48.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 -- 172.21.15.144:0/2656129004 shutdown_connections 2023-11-28T11:51:48.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.718+0000 7fea6afd4700 1 -- 172.21.15.144:0/2656129004 wait complete. 2023-11-28T11:51:48.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.719+0000 7fea6afd4700 1 Processor -- start 2023-11-28T11:51:48.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.719+0000 7fea6afd4700 1 -- start start 2023-11-28T11:51:48.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea6406ba90 0x7fea64158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 0x7fea640c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6415cef0 0x7fea64145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6415a0c0 con 0x7fea640cbdf0 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea640ca170 con 0x7fea6415cef0 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea64145f70 con 0x7fea6406ba90 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 0x7fea640c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 0x7fea640c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35576/0 (socket says 172.21.15.144:35576) 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 -- 172.21.15.144:0/3419173504 learned_addr learned my addr 172.21.15.144:0/3419173504 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea68d70700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea6406ba90 0x7fea64158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea69571700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6415cef0 0x7fea64145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 -- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea6406ba90 msgr2=0x7fea64158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea6406ba90 0x7fea64158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 -- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6415cef0 msgr2=0x7fea64145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6415cef0 0x7fea64145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 -- 172.21.15.144:0/3419173504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500094a0 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea69571700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6415cef0 0x7fea64145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea63fff700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 0x7fea640c9c30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea5000ebe0 tx=0x7fea50009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000c910 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5000ca70 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000cde0 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.720+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c009b30 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.721+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6415dff0 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.722+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6415dd10 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.722+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea50019030 con 0x7fea640cbdf0 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.722+0000 7fea617fa700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea480682f0 0x7fea4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:48.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.722+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea50085d50 con 0x7fea640cbdf0 2023-11-28T11:51:48.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.722+0000 7fea68d70700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea480682f0 0x7fea4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:48.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.723+0000 7fea68d70700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea480682f0 0x7fea4806a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea640c7710 tx=0x7fea58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:48.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.727+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea500525b0 con 0x7fea640cbdf0 2023-11-28T11:51:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.913+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea640cab90 con 0x7fea640cbdf0 2023-11-28T11:51:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.913+0000 7fea617fa700 1 -- 172.21.15.144:0/3419173504 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea50017020 con 0x7fea640cbdf0 2023-11-28T11:51:48.916 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea480682f0 msgr2=0x7fea4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea480682f0 0x7fea4806a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea640c7710 tx=0x7fea58009040 comp rx=0 tx=0).stop 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 msgr2=0x7fea640c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 0x7fea640c9c30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea5000ebe0 tx=0x7fea50009ec0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 shutdown_connections 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea480682f0 0x7fea4806a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6415cef0 0x7fea64145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea6406ba90 0x7fea64158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 --2- 172.21.15.144:0/3419173504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea640cbdf0 0x7fea640c9c30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:48.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 >> 172.21.15.144:0/3419173504 conn(0x7fea640d0000 msgr2=0x7fea640af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:48.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 shutdown_connections 2023-11-28T11:51:48.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:48.916+0000 7fea6afd4700 1 -- 172.21.15.144:0/3419173504 wait complete. 2023-11-28T11:51:48.930 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:49.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.053+0000 7f2760297700 1 Processor -- start 2023-11-28T11:51:49.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.053+0000 7f2760297700 1 -- start start 2023-11-28T11:51:49.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f2760297700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f275806c990 0x7f275806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f2760297700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275806ae60 con 0x7f275806e9f0 2023-11-28T11:51:49.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f2760297700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275806a270 con 0x7f27580ca240 2023-11-28T11:51:49.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f2760297700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275806d350 con 0x7f275806c990 2023-11-28T11:51:49.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f275806c990 0x7f275806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f275806c990 0x7f275806ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35586/0 (socket says 172.21.15.144:35586) 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 -- 172.21.15.144:0/2565150902 learned_addr learned my addr 172.21.15.144:0/2565150902 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 -- 172.21.15.144:0/2565150902 >> v1:172.21.15.177:6789/0 conn(0x7f27580ca240 legacy=0x7f27580ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 -- 172.21.15.144:0/2565150902 >> v1:172.21.15.144:6789/0 conn(0x7f275806e9f0 legacy=0x7f275806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 -- 172.21.15.144:0/2565150902 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275806a530 con 0x7f275806c990 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d832700 1 --2- 172.21.15.144:0/2565150902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f275806c990 0x7f275806ce10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f274800aa90 tx=0x7f274800ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb43b759a8698fd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.054+0000 7f275d031700 1 -- 172.21.15.144:0/2565150902 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274800cb30 con 0x7f275806c990 2023-11-28T11:51:49.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f275d031700 1 -- 172.21.15.144:0/2565150902 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274800cc90 con 0x7f275806c990 2023-11-28T11:51:49.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f275d031700 1 -- 172.21.15.144:0/2565150902 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748004030 con 0x7f275806c990 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 -- 172.21.15.144:0/2565150902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f275806c990 msgr2=0x7f275806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 --2- 172.21.15.144:0/2565150902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f275806c990 0x7f275806ce10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f274800aa90 tx=0x7f274800ada0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 -- 172.21.15.144:0/2565150902 shutdown_connections 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 --2- 172.21.15.144:0/2565150902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f275806c990 0x7f275806ce10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 -- 172.21.15.144:0/2565150902 >> 172.21.15.144:0/2565150902 conn(0x7f27580b12b0 msgr2=0x7f27580b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 -- 172.21.15.144:0/2565150902 shutdown_connections 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.055+0000 7f2760297700 1 -- 172.21.15.144:0/2565150902 wait complete. 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.057+0000 7f2760297700 1 Processor -- start 2023-11-28T11:51:49.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.057+0000 7f2760297700 1 -- start start 2023-11-28T11:51:49.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f275806c990 0x7f27581e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f275806e9f0 0x7f27581523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 0x7f27580c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758157480 con 0x7f27580ca240 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27581e0cd0 con 0x7f275806c990 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27581e0fb0 con 0x7f275806e9f0 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 0x7f27580c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 0x7f27580c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35592/0 (socket says 172.21.15.144:35592) 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 -- 172.21.15.144:0/1111911505 learned_addr learned my addr 172.21.15.144:0/1111911505 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:49.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275d832700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f275806e9f0 0x7f27581523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e033700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f275806c990 0x7f27581e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 -- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f275806e9f0 msgr2=0x7f27581523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f275806e9f0 0x7f27581523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 -- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f275806c990 msgr2=0x7f27581e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f275806c990 0x7f27581e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 -- 172.21.15.144:0/1111911505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274800a6f0 con 0x7f27580ca240 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e033700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f275806c990 0x7f27581e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f275e834700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 0x7f27580c8a70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f275000ee40 tx=0x7f275000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750011670 con 0x7f27580ca240 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500117d0 con 0x7f27580ca240 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275806aaf0 con 0x7f27580ca240 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750011b80 con 0x7f27580ca240 2023-11-28T11:51:49.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.058+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275806a810 con 0x7f27580ca240 2023-11-28T11:51:49.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.059+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275806ba00 con 0x7f27580ca240 2023-11-28T11:51:49.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.060+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2750011ce0 con 0x7f27580ca240 2023-11-28T11:51:49.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.060+0000 7f274effd700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f273c0682f0 0x7f273c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.060+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2750088920 con 0x7f27580ca240 2023-11-28T11:51:49.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.060+0000 7f275e033700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f273c0682f0 0x7f273c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.061+0000 7f275e033700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f273c0682f0 0x7f273c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2754001670 tx=0x7f2754009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:49.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.064+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2750055180 con 0x7f27580ca240 2023-11-28T11:51:49.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.246+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27580cd5b0 con 0x7f27580ca240 2023-11-28T11:51:49.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.246+0000 7f274effd700 1 -- 172.21.15.144:0/1111911505 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f275001f020 con 0x7f27580ca240 2023-11-28T11:51:49.248 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:49.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f273c0682f0 msgr2=0x7f273c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f273c0682f0 0x7f273c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2754001670 tx=0x7f2754009040 comp rx=0 tx=0).stop 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 msgr2=0x7f27580c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 0x7f27580c8a70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f275000ee40 tx=0x7f275000c550 comp rx=0 tx=0).stop 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 shutdown_connections 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27580ca240 0x7f27580c8a70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f273c0682f0 0x7f273c06a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f275806c990 0x7f27581e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 --2- 172.21.15.144:0/1111911505 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f275806e9f0 0x7f27581523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.248+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 >> 172.21.15.144:0/1111911505 conn(0x7f27580b12b0 msgr2=0x7f27580ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.249+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 shutdown_connections 2023-11-28T11:51:49.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.249+0000 7f2760297700 1 -- 172.21.15.144:0/1111911505 wait complete. 2023-11-28T11:51:49.263 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:51:49.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.382+0000 7fede4283700 1 Processor -- start 2023-11-28T11:51:49.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.382+0000 7fede4283700 1 -- start start 2023-11-28T11:51:49.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede4283700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feddc06e9f0 0x7feddc06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede4283700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feddc06b830 0x7feddc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede4283700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc06ae60 con 0x7feddc06ced0 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede4283700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc06d2c0 con 0x7feddc06e9f0 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede4283700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc06a270 con 0x7feddc06b830 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede201f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feddc06e9f0 0x7feddc06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede201f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feddc06e9f0 0x7feddc06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35596/0 (socket says 172.21.15.144:35596) 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede201f700 1 -- 172.21.15.144:0/3753102452 learned_addr learned my addr 172.21.15.144:0/3753102452 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.383+0000 7fede181e700 1 --2- 172.21.15.144:0/3753102452 >> v2:172.21.15.177:3300/0 conn(0x7feddc06b830 0x7feddc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede201f700 1 -- 172.21.15.144:0/3753102452 >> v2:172.21.15.177:3300/0 conn(0x7feddc06b830 msgr2=0x7feddc06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede201f700 1 --2- 172.21.15.144:0/3753102452 >> v2:172.21.15.177:3300/0 conn(0x7feddc06b830 0x7feddc06c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede201f700 1 -- 172.21.15.144:0/3753102452 >> v1:172.21.15.177:6789/0 conn(0x7feddc06ced0 legacy=0x7feddc0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede201f700 1 -- 172.21.15.144:0/3753102452 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc06a530 con 0x7feddc06e9f0 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede201f700 1 --2- 172.21.15.144:0/3753102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06e9f0 0x7feddc06b2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feddc0af1d0 tx=0x7fedd8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d0dfd530ed596f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:49.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede101d700 1 -- 172.21.15.144:0/3753102452 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004230 con 0x7feddc06e9f0 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede101d700 1 -- 172.21.15.144:0/3753102452 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8004390 con 0x7feddc06e9f0 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede101d700 1 -- 172.21.15.144:0/3753102452 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd80036a0 con 0x7feddc06e9f0 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 -- 172.21.15.144:0/3753102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06e9f0 msgr2=0x7feddc06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 --2- 172.21.15.144:0/3753102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06e9f0 0x7feddc06b2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feddc0af1d0 tx=0x7fedd8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 -- 172.21.15.144:0/3753102452 shutdown_connections 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 --2- 172.21.15.144:0/3753102452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06e9f0 0x7feddc06b2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 --2- 172.21.15.144:0/3753102452 >> v2:172.21.15.177:3300/0 conn(0x7feddc06b830 0x7feddc06c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 -- 172.21.15.144:0/3753102452 >> 172.21.15.144:0/3753102452 conn(0x7feddc0b12b0 msgr2=0x7feddc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.384+0000 7fede4283700 1 -- 172.21.15.144:0/3753102452 shutdown_connections 2023-11-28T11:51:49.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.385+0000 7fede4283700 1 -- 172.21.15.144:0/3753102452 wait complete. 2023-11-28T11:51:49.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.386+0000 7fede4283700 1 Processor -- start 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.386+0000 7fede4283700 1 -- start start 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede4283700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feddc06b830 0x7feddc1e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede4283700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 0x7feddc1497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede4283700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feddc06e9f0 0x7feddc149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede4283700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1e0490 con 0x7feddc06ced0 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede4283700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1e0d30 con 0x7feddc06e9f0 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede4283700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0c8080 con 0x7feddc06b830 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 0x7feddc1497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 0x7feddc1497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35602/0 (socket says 172.21.15.144:35602) 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 -- 172.21.15.144:0/1890376199 learned_addr learned my addr 172.21.15.144:0/1890376199 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede201f700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feddc06b830 0x7feddc1e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede2820700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feddc06e9f0 0x7feddc149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 -- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feddc06b830 msgr2=0x7feddc1e2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feddc06b830 0x7feddc1e2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 -- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feddc06e9f0 msgr2=0x7feddc149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feddc06e9f0 0x7feddc149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 -- 172.21.15.144:0/1890376199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8009550 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.387+0000 7fede181e700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 0x7feddc1497f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00d830 tx=0x7fedcc00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.388+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0125f0 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.388+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc012750 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.388+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc012b00 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.388+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc06aaf0 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.388+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc06a810 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.389+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc06edc0 con 0x7feddc06ced0 2023-11-28T11:51:49.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.389+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedcc012c60 con 0x7feddc06ced0 2023-11-28T11:51:49.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.389+0000 7fedd2ffd700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedc00682f0 0x7fedc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:49.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.389+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fedcc086190 con 0x7feddc06ced0 2023-11-28T11:51:49.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.390+0000 7fede201f700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedc00682f0 0x7fedc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:49.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.390+0000 7fede201f700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedc00682f0 0x7fedc006a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009130 tx=0x7fedd8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:49.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.394+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedcc0529f0 con 0x7feddc06ced0 2023-11-28T11:51:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.591+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feddc1e0e70 con 0x7feddc06ced0 2023-11-28T11:51:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.591+0000 7fedd2ffd700 1 -- 172.21.15.144:0/1890376199 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fedcc056010 con 0x7feddc06ced0 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedc00682f0 msgr2=0x7fedc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedc00682f0 0x7fedc006a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009130 tx=0x7fedd8008030 comp rx=0 tx=0).stop 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 msgr2=0x7feddc1497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 0x7feddc1497f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00d830 tx=0x7fedcc00dbf0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 shutdown_connections 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fedc00682f0 0x7fedc006a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feddc06e9f0 0x7feddc149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feddc06b830 0x7feddc1e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 --2- 172.21.15.144:0/1890376199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc06ced0 0x7feddc1497f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:49.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 >> 172.21.15.144:0/1890376199 conn(0x7feddc0b12b0 msgr2=0x7feddc0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:49.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 shutdown_connections 2023-11-28T11:51:49.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:49.593+0000 7fede4283700 1 -- 172.21.15.144:0/1890376199 wait complete. 2023-11-28T11:51:49.596 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:51:49.607 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T11:51:49.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:49.607 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T11:51:49.652 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:51:49.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:49.652 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:51:49.723 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:49.724 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:49.741 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:49.741 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:49.799 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:51:49.799 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:49.857 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:50.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.015+0000 7f5c7e0e7700 1 Processor -- start 2023-11-28T11:51:50.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.015+0000 7f5c7e0e7700 1 -- start start 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c7e0e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5c78070c50 0x7f5c780b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c7e0e7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7815e1a0 con 0x7f5c78158200 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c7e0e7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780b0fd0 con 0x7f5c780b0bb0 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c7e0e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7815f070 con 0x7f5c78070c50 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c777fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5c78158200 0x7f5c780caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52320/0 (socket says 172.21.15.144:52320) 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c777fe700 1 -- 172.21.15.144:0/870175082 learned_addr learned my addr 172.21.15.144:0/870175082 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c76ffd700 1 --2- 172.21.15.144:0/870175082 >> v2:172.21.15.177:3300/0 conn(0x7f5c78070c50 0x7f5c780b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362561137 0 0) 0x7f5c7815e1a0 con 0x7f5c78158200 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c600035b0 con 0x7f5c78158200 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550173184 0 0) 0x7f5c600035b0 con 0x7f5c78158200 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 >> v2:172.21.15.177:3300/0 conn(0x7f5c78070c50 msgr2=0x7f5c780b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 --2- 172.21.15.144:0/870175082 >> v2:172.21.15.177:3300/0 conn(0x7f5c78070c50 0x7f5c780b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 >> v1:172.21.15.177:6789/0 conn(0x7f5c780b0bb0 legacy=0x7f5c7815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780cb180 con 0x7f5c78158200 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f5c6c001670 con 0x7f5c78158200 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 >> v1:172.21.15.144:6789/0 conn(0x7f5c78158200 legacy=0x7f5c780caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c60004850 0x7f5c60004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c60005200 0x7f5c6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815f070 con 0x7f5c60003a00 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6000c110 con 0x7f5c60004850 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.016+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6000c2d0 con 0x7f5c60005200 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c76ffd700 1 --2- 172.21.15.144:0/870175082 >> v2:172.21.15.177:3300/0 conn(0x7f5c78070c50 0x7f5c780b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c77fff700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c60005200 0x7f5c6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c76ffd700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c60004850 0x7f5c60004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 -- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c60005200 msgr2=0x7f5c6000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c60005200 0x7f5c6000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 -- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c60004850 msgr2=0x7f5c60004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c60004850 0x7f5c60004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 -- 172.21.15.144:0/870175082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600035b0 con 0x7f5c60003a00 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c77fff700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c60005200 0x7f5c6000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c77fff700 1 -- 172.21.15.144:0/870175082 reap_dead start 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c777fe700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c60003e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c2a0 tx=0x7f5c60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ab4ebad8d1dcaa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c007c10 con 0x7f5c60003a00 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c00cde0 con 0x7f5c60003a00 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.017+0000 7f5c767fc700 1 -- 172.21.15.144:0/870175082 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c005ba0 con 0x7f5c60003a00 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/870175082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 msgr2=0x7f5c60003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c60003e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c2a0 tx=0x7f5c60005880 comp rx=0 tx=0).stop 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/870175082 shutdown_connections 2023-11-28T11:51:50.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/870175082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c60003e10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/870175082 >> 172.21.15.144:0/870175082 conn(0x7f5c780cb670 msgr2=0x7f5c780cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/870175082 shutdown_connections 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.018+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/870175082 wait complete. 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.019+0000 7f5c7e0e7700 1 Processor -- start 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.019+0000 7f5c7e0e7700 1 -- start start 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c78158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c780c86d0 0x7f5c781590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7815be00 0x7f5c7815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780b2f40 con 0x7f5c60003a00 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815cb30 con 0x7f5c7815be00 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815ce10 con 0x7f5c780c86d0 2023-11-28T11:51:50.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c78158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c78158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35620/0 (socket says 172.21.15.144:35620) 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 -- 172.21.15.144:0/1480665960 learned_addr learned my addr 172.21.15.144:0/1480665960 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c77fff700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7815be00 0x7f5c7815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c76ffd700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c780c86d0 0x7f5c781590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 -- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c780c86d0 msgr2=0x7f5c781590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c780c86d0 0x7f5c781590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 -- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7815be00 msgr2=0x7f5c7815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7815be00 0x7f5c7815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 -- 172.21.15.144:0/1480665960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c781585d0 con 0x7f5c60003a00 2023-11-28T11:51:50.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c77fff700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7815be00 0x7f5c7815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c777fe700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c78158b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000c0e0 tx=0x7f5c7806fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c001630 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c0070d0 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c007480 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.020+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780710f0 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.021+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7815e3f0 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.022+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c780b3220 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.022+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c6c02d070 con 0x7f5c60003a00 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.022+0000 7f5c5ffff700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.022+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c6c087200 con 0x7f5c60003a00 2023-11-28T11:51:50.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.022+0000 7f5c76ffd700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.023+0000 7f5c76ffd700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c640682f0 0x7f5c6406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c780b7100 tx=0x7f5c68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.027+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c6c0539e0 con 0x7f5c60003a00 2023-11-28T11:51:50.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.213+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c7806fd60 con 0x7f5c60003a00 2023-11-28T11:51:50.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.214+0000 7f5c5ffff700 1 -- 172.21.15.144:0/1480665960 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f5c6c057000 con 0x7f5c60003a00 2023-11-28T11:51:50.216 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:50.216 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:50.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c640682f0 msgr2=0x7f5c6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c640682f0 0x7f5c6406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c780b7100 tx=0x7f5c68008040 comp rx=0 tx=0).stop 2023-11-28T11:51:50.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 msgr2=0x7f5c78158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c78158b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000c0e0 tx=0x7f5c7806fc40 comp rx=0 tx=0).stop 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 shutdown_connections 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c60003a00 0x7f5c78158b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c7815be00 0x7f5c7815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 --2- 172.21.15.144:0/1480665960 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c780c86d0 0x7f5c781590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 >> 172.21.15.144:0/1480665960 conn(0x7f5c780cb670 msgr2=0x7f5c78156280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 shutdown_connections 2023-11-28T11:51:50.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.216+0000 7f5c7e0e7700 1 -- 172.21.15.144:0/1480665960 wait complete. 2023-11-28T11:51:50.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:50.232 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:50.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.352+0000 7f102bf26700 1 Processor -- start 2023-11-28T11:51:50.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.353+0000 7f102bf26700 1 -- start start 2023-11-28T11:51:50.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.353+0000 7f102bf26700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1024071040 0x7f1024071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.353+0000 7f102bf26700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1024071950 con 0x7f102406dee0 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.353+0000 7f102bf26700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102415d060 con 0x7f1024160540 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.353+0000 7f102bf26700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102415d340 con 0x7f1024071040 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f10294c1700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f102406dee0 0x7f102406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52340/0 (socket says 172.21.15.144:52340) 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f10294c1700 1 -- 172.21.15.144:0/1586797438 learned_addr learned my addr 172.21.15.144:0/1586797438 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1029cc2700 1 --2- 172.21.15.144:0/1586797438 >> v2:172.21.15.177:3300/0 conn(0x7f1024071040 0x7f1024071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900740446 0 0) 0x7f1024071950 con 0x7f102406dee0 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10100035b0 con 0x7f102406dee0 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360539153 0 0) 0x7f102415d060 con 0x7f1024160540 2023-11-28T11:51:50.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1024071950 con 0x7f1024160540 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374544573 0 0) 0x7f10100035b0 con 0x7f102406dee0 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 >> v2:172.21.15.177:3300/0 conn(0x7f1024071040 msgr2=0x7f1024071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 --2- 172.21.15.144:0/1586797438 >> v2:172.21.15.177:3300/0 conn(0x7f1024071040 0x7f1024071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 >> v1:172.21.15.177:6789/0 conn(0x7f1024160540 legacy=0x7f1024160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102415d620 con 0x7f102406dee0 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f1020001670 con 0x7f102406dee0 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1029cc2700 1 --2- 172.21.15.144:0/1586797438 >> v2:172.21.15.177:3300/0 conn(0x7f1024071040 0x7f1024071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 >> v1:172.21.15.144:6789/0 conn(0x7f102406dee0 legacy=0x7f102406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1010003b90 0x7f1010003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1010004850 0x7f1010004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010005270 0x7f101000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102415d340 con 0x7f1010004850 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101000c1b0 con 0x7f1010003b90 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101000c370 con 0x7f1010005270 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f10294c1700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1010004850 0x7f1010004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f1029cc2700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1010003b90 0x7f1010003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.354+0000 7f102a4c3700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010005270 0x7f101000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f10294c1700 1 -- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010005270 msgr2=0x7f101000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f10294c1700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010005270 0x7f101000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f10294c1700 1 -- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1010003b90 msgr2=0x7f1010003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f10294c1700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1010003b90 0x7f1010003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f10294c1700 1 -- 172.21.15.144:0/1586797438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100035b0 con 0x7f1010004850 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102a4c3700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010005270 0x7f101000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:50.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102a4c3700 1 -- 172.21.15.144:0/1586797438 reap_dead start 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f10294c1700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1010004850 0x7f1010004cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f102000c210 tx=0x7f1010005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76ef2d24c56b1b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000cdb0 con 0x7f1010004850 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020001af0 con 0x7f1010004850 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f1028cc0700 1 -- 172.21.15.144:0/1586797438 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020005ba0 con 0x7f1010004850 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102bf26700 1 -- 172.21.15.144:0/1586797438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1010004850 msgr2=0x7f1010004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102bf26700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1010004850 0x7f1010004cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f102000c210 tx=0x7f1010005880 comp rx=0 tx=0).stop 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102bf26700 1 -- 172.21.15.144:0/1586797438 shutdown_connections 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102bf26700 1 --2- 172.21.15.144:0/1586797438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1010004850 0x7f1010004cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.355+0000 7f102bf26700 1 -- 172.21.15.144:0/1586797438 >> 172.21.15.144:0/1586797438 conn(0x7f10240d0000 msgr2=0x7f10240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.356+0000 7f102bf26700 1 -- 172.21.15.144:0/1586797438 shutdown_connections 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.356+0000 7f102bf26700 1 -- 172.21.15.144:0/1586797438 wait complete. 2023-11-28T11:51:50.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.357+0000 7f102bf26700 1 Processor -- start 2023-11-28T11:51:50.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.357+0000 7f102bf26700 1 -- start start 2023-11-28T11:51:50.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102bf26700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010004850 0x7f10240b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102bf26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 0x7f10241f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102bf26700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10240b57f0 0x7f10240ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102bf26700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241f33c0 con 0x7f102414c460 2023-11-28T11:51:50.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102bf26700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240c5670 con 0x7f10240b57f0 2023-11-28T11:51:50.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102bf26700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10240c4d90 con 0x7f1010004850 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 0x7f10241f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 0x7f10241f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35642/0 (socket says 172.21.15.144:35642) 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 -- 172.21.15.144:0/185939265 learned_addr learned my addr 172.21.15.144:0/185939265 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 -- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010004850 msgr2=0x7f10240b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010004850 0x7f10240b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 -- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10240b57f0 msgr2=0x7f10240ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f102a4c3700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10240b57f0 0x7f10240ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10240b57f0 0x7f10240ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.358+0000 7f10294c1700 1 -- 172.21.15.144:0/185939265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102415ccd0 con 0x7f102414c460 2023-11-28T11:51:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.359+0000 7f10294c1700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 0x7f10241f2e80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f101000c180 tx=0x7f1020001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.359+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200070d0 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.359+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020007230 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.359+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102415d900 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.359+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200075e0 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.359+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102406a270 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.360+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10240720a0 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.360+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1020010070 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.360+0000 7f101a7fc700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10080682f0 0x7f100806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.360+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1020086490 con 0x7f102414c460 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.360+0000 7f1029cc2700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10080682f0 0x7f100806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.361+0000 7f1029cc2700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10080682f0 0x7f100806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1014006dd0 tx=0x7f1014006d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.365+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1020052cf0 con 0x7f102414c460 2023-11-28T11:51:50.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.545+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f102415fa40 con 0x7f102414c460 2023-11-28T11:51:50.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.546+0000 7f101a7fc700 1 -- 172.21.15.144:0/185939265 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1020056310 con 0x7f102414c460 2023-11-28T11:51:50.548 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10080682f0 msgr2=0x7f100806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10080682f0 0x7f100806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1014006dd0 tx=0x7f1014006d40 comp rx=0 tx=0).stop 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 msgr2=0x7f10241f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 0x7f10241f2e80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f101000c180 tx=0x7f1020001980 comp rx=0 tx=0).stop 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 shutdown_connections 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10080682f0 0x7f100806a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10240b57f0 0x7f10240ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1010004850 0x7f10240b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 --2- 172.21.15.144:0/185939265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102414c460 0x7f10241f2e80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 >> 172.21.15.144:0/185939265 conn(0x7f10240d0000 msgr2=0x7f102406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:50.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 shutdown_connections 2023-11-28T11:51:50.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.548+0000 7f102bf26700 1 -- 172.21.15.144:0/185939265 wait complete. 2023-11-28T11:51:50.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:50.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.684+0000 7fd6bf860700 1 Processor -- start 2023-11-28T11:51:50.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.684+0000 7fd6bf860700 1 -- start start 2023-11-28T11:51:50.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bf860700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd6b80d01b0 0x7fd6b80d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bf860700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b20b0 0x7fd6b80d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bf860700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6b80b29b0 0x7fd6b80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bf860700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b815cec0 con 0x7fd6b80b29b0 2023-11-28T11:51:50.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bf860700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b80b0230 con 0x7fd6b80d01b0 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bf860700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80b4570 con 0x7fd6b80b20b0 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6b80b29b0 0x7fd6b80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6b80b29b0 0x7fd6b80b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35652/0 (socket says 172.21.15.144:35652) 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 -- 172.21.15.144:0/3156509971 learned_addr learned my addr 172.21.15.144:0/3156509971 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bd5fc700 1 --2- 172.21.15.144:0/3156509971 >> v2:172.21.15.177:3300/0 conn(0x7fd6b80d01b0 0x7fd6b80d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bcdfb700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b20b0 0x7fd6b80d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 -- 172.21.15.144:0/3156509971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b20b0 msgr2=0x7fd6b80d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b20b0 0x7fd6b80d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 -- 172.21.15.144:0/3156509971 >> v2:172.21.15.177:3300/0 conn(0x7fd6b80d01b0 msgr2=0x7fd6b80d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 --2- 172.21.15.144:0/3156509971 >> v2:172.21.15.177:3300/0 conn(0x7fd6b80d01b0 0x7fd6b80d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 -- 172.21.15.144:0/3156509971 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b80b4850 con 0x7fd6b80b29b0 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bcdfb700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b20b0 0x7fd6b80d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:50.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bd5fc700 1 --2- 172.21.15.144:0/3156509971 >> v2:172.21.15.177:3300/0 conn(0x7fd6b80d01b0 0x7fd6b80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6bddfd700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b29b0 0x7fd6b80b2e30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80b5bb0 tx=0x7fd6b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56316816328cf904 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6affff700 1 -- 172.21.15.144:0/3156509971 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000e070 con 0x7fd6b80b29b0 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6affff700 1 -- 172.21.15.144:0/3156509971 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000b9c0 con 0x7fd6b80b29b0 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.685+0000 7fd6affff700 1 -- 172.21.15.144:0/3156509971 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000bd70 con 0x7fd6b80b29b0 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 -- 172.21.15.144:0/3156509971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b29b0 msgr2=0x7fd6b80b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b29b0 0x7fd6b80b2e30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80b5bb0 tx=0x7fd6b000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 -- 172.21.15.144:0/3156509971 shutdown_connections 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b29b0 0x7fd6b80b2e30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 --2- 172.21.15.144:0/3156509971 >> v2:172.21.15.177:3300/0 conn(0x7fd6b80d01b0 0x7fd6b80d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 --2- 172.21.15.144:0/3156509971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b20b0 0x7fd6b80d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 -- 172.21.15.144:0/3156509971 >> 172.21.15.144:0/3156509971 conn(0x7fd6b809c3f0 msgr2=0x7fd6b80d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:50.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 -- 172.21.15.144:0/3156509971 shutdown_connections 2023-11-28T11:51:50.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.686+0000 7fd6bf860700 1 -- 172.21.15.144:0/3156509971 wait complete. 2023-11-28T11:51:50.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.687+0000 7fd6bf860700 1 Processor -- start 2023-11-28T11:51:50.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.687+0000 7fd6bf860700 1 -- start start 2023-11-28T11:51:50.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bf860700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 0x7fd6b81f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bf860700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b29b0 0x7fd6b80b8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bf860700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6b80d01b0 0x7fd6b80b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bf860700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80c6ac0 con 0x7fd6b80b20b0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bf860700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80c6da0 con 0x7fd6b80d01b0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bf860700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80c7080 con 0x7fd6b80b29b0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 0x7fd6b81f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 0x7fd6b81f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35654/0 (socket says 172.21.15.144:35654) 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 -- 172.21.15.144:0/1249891589 learned_addr learned my addr 172.21.15.144:0/1249891589 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bcdfb700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b29b0 0x7fd6b80b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bddfd700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6b80d01b0 0x7fd6b80b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 -- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b29b0 msgr2=0x7fd6b80b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b29b0 0x7fd6b80b8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 -- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6b80d01b0 msgr2=0x7fd6b80b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6b80d01b0 0x7fd6b80b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bd5fc700 1 -- 172.21.15.144:0/1249891589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b0009b30 con 0x7fd6b80b20b0 2023-11-28T11:51:50.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.688+0000 7fd6bcdfb700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b29b0 0x7fd6b80b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.689+0000 7fd6bd5fc700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 0x7fd6b81f2690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c7f0 tx=0x7fd6b400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.689+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4011800 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.689+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b4011960 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.689+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b80afec0 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.689+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4011d10 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.689+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b80afbe0 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.690+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b80b2600 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.690+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6b4021070 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.690+0000 7fd6adffb700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd69c0682f0 0x7fd69c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.691+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6b4085b10 con 0x7fd6b80b20b0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.691+0000 7fd6bcdfb700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd69c0682f0 0x7fd69c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:50.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.691+0000 7fd6bcdfb700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd69c0682f0 0x7fd69c06a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006ba0 tx=0x7fd6a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:50.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.695+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6b4052370 con 0x7fd6b80b20b0 2023-11-28T11:51:50.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.878+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6b8068690 con 0x7fd6b80b20b0 2023-11-28T11:51:50.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.878+0000 7fd6adffb700 1 -- 172.21.15.144:0/1249891589 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6b4055990 con 0x7fd6b80b20b0 2023-11-28T11:51:50.880 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:50.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd69c0682f0 msgr2=0x7fd69c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd69c0682f0 0x7fd69c06a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8006ba0 tx=0x7fd6a8008040 comp rx=0 tx=0).stop 2023-11-28T11:51:50.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 msgr2=0x7fd6b81f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:50.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 0x7fd6b81f2690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c7f0 tx=0x7fd6b400cbb0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 shutdown_connections 2023-11-28T11:51:50.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6b80b20b0 0x7fd6b81f2690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd69c0682f0 0x7fd69c06a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6b80d01b0 0x7fd6b80b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 --2- 172.21.15.144:0/1249891589 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6b80b29b0 0x7fd6b80b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:50.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 >> 172.21.15.144:0/1249891589 conn(0x7fd6b809c3f0 msgr2=0x7fd6b80d2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:50.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 shutdown_connections 2023-11-28T11:51:50.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:50.880+0000 7fd6bf860700 1 -- 172.21.15.144:0/1249891589 wait complete. 2023-11-28T11:51:50.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:51.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.016+0000 7f189f609700 1 Processor -- start 2023-11-28T11:51:51.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.016+0000 7f189f609700 1 -- start start 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189f609700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f18980d0a00 0x7f189815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189f609700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f18980b2930 0x7f18980b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189f609700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189806b420 con 0x7f18980ce040 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189f609700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18980b32f0 con 0x7f18980b2930 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189f609700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189806ff60 con 0x7f18980d0a00 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189d3a5700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f18980ce040 0x7f189806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52374/0 (socket says 172.21.15.144:52374) 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189d3a5700 1 -- 172.21.15.144:0/3500224709 learned_addr learned my addr 172.21.15.144:0/3500224709 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189dba6700 1 --2- 172.21.15.144:0/3500224709 >> v2:172.21.15.144:3300/0 conn(0x7f18980b2930 0x7f18980b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189cba4700 1 --2- 172.21.15.144:0/3500224709 >> v2:172.21.15.177:3300/0 conn(0x7f18980d0a00 0x7f189815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189dba6700 1 -- 172.21.15.144:0/3500224709 >> v2:172.21.15.177:3300/0 conn(0x7f18980d0a00 msgr2=0x7f189815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189dba6700 1 --2- 172.21.15.144:0/3500224709 >> v2:172.21.15.177:3300/0 conn(0x7f18980d0a00 0x7f189815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189dba6700 1 -- 172.21.15.144:0/3500224709 >> v1:172.21.15.144:6789/0 conn(0x7f18980ce040 legacy=0x7f189806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189dba6700 1 -- 172.21.15.144:0/3500224709 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189806b700 con 0x7f18980b2930 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f189dba6700 1 --2- 172.21.15.144:0/3500224709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980b2930 0x7f18980b2db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f189806ed50 tx=0x7f189000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5bdbf4481d1990 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f188ffff700 1 -- 172.21.15.144:0/3500224709 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000e070 con 0x7f18980b2930 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f188ffff700 1 -- 172.21.15.144:0/3500224709 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189000bb20 con 0x7f18980b2930 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.017+0000 7f188ffff700 1 -- 172.21.15.144:0/3500224709 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890012430 con 0x7f18980b2930 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 -- 172.21.15.144:0/3500224709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980b2930 msgr2=0x7f18980b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 --2- 172.21.15.144:0/3500224709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980b2930 0x7f18980b2db0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f189806ed50 tx=0x7f189000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 -- 172.21.15.144:0/3500224709 shutdown_connections 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 --2- 172.21.15.144:0/3500224709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980b2930 0x7f18980b2db0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 --2- 172.21.15.144:0/3500224709 >> v2:172.21.15.177:3300/0 conn(0x7f18980d0a00 0x7f189815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 -- 172.21.15.144:0/3500224709 >> 172.21.15.144:0/3500224709 conn(0x7f189806aed0 msgr2=0x7f189806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:51.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 -- 172.21.15.144:0/3500224709 shutdown_connections 2023-11-28T11:51:51.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.018+0000 7f189f609700 1 -- 172.21.15.144:0/3500224709 wait complete. 2023-11-28T11:51:51.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.020+0000 7f189f609700 1 Processor -- start 2023-11-28T11:51:51.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.020+0000 7f189f609700 1 -- start start 2023-11-28T11:51:51.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.020+0000 7f189f609700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18980b2930 0x7f18980c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 0x7f18981f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18980d0a00 0x7f18981f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b6d50 con 0x7f18980ce040 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b3e30 con 0x7f18980d0a00 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b4110 con 0x7f18980b2930 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 0x7f18981f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 0x7f18981f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35670/0 (socket says 172.21.15.144:35670) 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 -- 172.21.15.144:0/1524121045 learned_addr learned my addr 172.21.15.144:0/1524121045 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189d3a5700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18980b2930 0x7f18980c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 -- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18980b2930 msgr2=0x7f18980c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18980b2930 0x7f18980c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 -- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18980d0a00 msgr2=0x7f18981f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189dba6700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18980d0a00 0x7f18981f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18980d0a00 0x7f18981f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 -- 172.21.15.144:0/1524121045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890009a70 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189cba4700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 0x7f18981f5d70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f188800e870 tx=0x7f188800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888009590 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188800ee70 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189806bcc0 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800ca50 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.021+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189806b9e0 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.022+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189806bfa0 con 0x7f18980ce040 2023-11-28T11:51:51.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.023+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1888019030 con 0x7f18980ce040 2023-11-28T11:51:51.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.023+0000 7f188dffb700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f187c0682f0 0x7f187c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.023+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1888085b20 con 0x7f18980ce040 2023-11-28T11:51:51.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.023+0000 7f189d3a5700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f187c0682f0 0x7f187c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.024+0000 7f189d3a5700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f187c0682f0 0x7f187c06a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18940010e0 tx=0x7f1894009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.027+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1888052380 con 0x7f18980ce040 2023-11-28T11:51:51.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.208+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18980c7f60 con 0x7f18980ce040 2023-11-28T11:51:51.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.209+0000 7f188dffb700 1 -- 172.21.15.144:0/1524121045 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18880559a0 con 0x7f18980ce040 2023-11-28T11:51:51.211 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:51.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.210+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f187c0682f0 msgr2=0x7f187c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.210+0000 7f189f609700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f187c0682f0 0x7f187c06a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18940010e0 tx=0x7f1894009040 comp rx=0 tx=0).stop 2023-11-28T11:51:51.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.210+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 msgr2=0x7f18981f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.210+0000 7f189f609700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 0x7f18981f5d70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f188800e870 tx=0x7f188800ec30 comp rx=0 tx=0).stop 2023-11-28T11:51:51.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 shutdown_connections 2023-11-28T11:51:51.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f187c0682f0 0x7f187c06a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18980d0a00 0x7f18981f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f18980b2930 0x7f18980c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 --2- 172.21.15.144:0/1524121045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18980ce040 0x7f18981f5d70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 >> 172.21.15.144:0/1524121045 conn(0x7f189806aed0 msgr2=0x7f189806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:51.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 shutdown_connections 2023-11-28T11:51:51.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.211+0000 7f189f609700 1 -- 172.21.15.144:0/1524121045 wait complete. 2023-11-28T11:51:51.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:51.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.348+0000 7ff20b153700 1 Processor -- start 2023-11-28T11:51:51.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.348+0000 7ff20b153700 1 -- start start 2023-11-28T11:51:51.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff20b153700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff2040cd8c0 0x7ff204140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff20b153700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff2040cdc90 0x7ff2040b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff20b153700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20413ff00 con 0x7ff2040b2e70 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff20b153700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2041404c0 con 0x7ff2040cdc90 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff20b153700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20415aa80 con 0x7ff2040cd8c0 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff203fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff2040cdc90 0x7ff2040b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff2096f0700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff2040b2e70 0x7ff20406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52392/0 (socket says 172.21.15.144:52392) 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff203fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff2040cdc90 0x7ff2040b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35678/0 (socket says 172.21.15.144:35678) 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff2096f0700 1 -- 172.21.15.144:0/3613688151 learned_addr learned my addr 172.21.15.144:0/3613688151 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.349+0000 7ff208eef700 1 --2- 172.21.15.144:0/3613688151 >> v2:172.21.15.177:3300/0 conn(0x7ff2040cd8c0 0x7ff204140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff203fff700 1 -- 172.21.15.144:0/3613688151 >> v2:172.21.15.177:3300/0 conn(0x7ff2040cd8c0 msgr2=0x7ff204140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff203fff700 1 --2- 172.21.15.144:0/3613688151 >> v2:172.21.15.177:3300/0 conn(0x7ff2040cd8c0 0x7ff204140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff203fff700 1 -- 172.21.15.144:0/3613688151 >> v1:172.21.15.144:6789/0 conn(0x7ff2040b2e70 legacy=0x7ff20406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff203fff700 1 -- 172.21.15.144:0/3613688151 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040b3310 con 0x7ff2040cdc90 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff2037fe700 1 -- 172.21.15.144:0/3613688151 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514748410 0 0) 0x7ff20413ff00 con 0x7ff2040b2e70 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff203fff700 1 --2- 172.21.15.144:0/3613688151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cdc90 0x7ff2040b2930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff204070a20 tx=0x7ff1f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4c2cbe74ebbd60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff2037fe700 1 -- 172.21.15.144:0/3613688151 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0011830 con 0x7ff2040cdc90 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff2037fe700 1 -- 172.21.15.144:0/3613688151 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f0011990 con 0x7ff2040cdc90 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.350+0000 7ff2037fe700 1 -- 172.21.15.144:0/3613688151 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0011d20 con 0x7ff2040cdc90 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 -- 172.21.15.144:0/3613688151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cdc90 msgr2=0x7ff2040b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 --2- 172.21.15.144:0/3613688151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cdc90 0x7ff2040b2930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff204070a20 tx=0x7ff1f0009b20 comp rx=0 tx=0).stop 2023-11-28T11:51:51.355 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 -- 172.21.15.144:0/3613688151 shutdown_connections 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 --2- 172.21.15.144:0/3613688151 >> v2:172.21.15.177:3300/0 conn(0x7ff2040cd8c0 0x7ff204140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 --2- 172.21.15.144:0/3613688151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cdc90 0x7ff2040b2930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 -- 172.21.15.144:0/3613688151 >> 172.21.15.144:0/3613688151 conn(0x7ff20406aed0 msgr2=0x7ff20406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 -- 172.21.15.144:0/3613688151 shutdown_connections 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.351+0000 7ff20b153700 1 -- 172.21.15.144:0/3613688151 wait complete. 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.352+0000 7ff20b153700 1 Processor -- start 2023-11-28T11:51:51.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.352+0000 7ff20b153700 1 -- start start 2023-11-28T11:51:51.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff20b153700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2040b2e70 0x7ff2041f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff20b153700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 0x7ff2040c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff20b153700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2040cdc90 0x7ff2040c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff20b153700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204143080 con 0x7ff2040cd8c0 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff20b153700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204143360 con 0x7ff2040cdc90 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff20b153700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff204143640 con 0x7ff2040b2e70 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 0x7ff2040c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 0x7ff2040c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35684/0 (socket says 172.21.15.144:35684) 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 -- 172.21.15.144:0/675703697 learned_addr learned my addr 172.21.15.144:0/675703697 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff208eef700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2040b2e70 0x7ff2041f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff2096f0700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2040cdc90 0x7ff2040c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 -- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2040b2e70 msgr2=0x7ff2041f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2040b2e70 0x7ff2041f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 -- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2040cdc90 msgr2=0x7ff2040c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2040cdc90 0x7ff2040c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 -- 172.21.15.144:0/675703697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00094a0 con 0x7ff2040cd8c0 2023-11-28T11:51:51.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff2096f0700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2040cdc90 0x7ff2040c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:51.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff203fff700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 0x7ff2040c6270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009b60 tx=0x7ff1f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00036a0 con 0x7ff2040cd8c0 2023-11-28T11:51:51.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.353+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f001e070 con 0x7ff2040cd8c0 2023-11-28T11:51:51.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.354+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff204140150 con 0x7ff2040cd8c0 2023-11-28T11:51:51.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.354+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0003a20 con 0x7ff2040cd8c0 2023-11-28T11:51:51.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.354+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2040b35f0 con 0x7ff2040cd8c0 2023-11-28T11:51:51.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.354+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2040ce100 con 0x7ff2040cd8c0 2023-11-28T11:51:51.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.355+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff1f000f050 con 0x7ff2040cd8c0 2023-11-28T11:51:51.363 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.355+0000 7ff2017fa700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff1e80682f0 0x7ff1e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.355+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff1f0086bc0 con 0x7ff2040cd8c0 2023-11-28T11:51:51.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.355+0000 7ff208eef700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff1e80682f0 0x7ff1e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.356+0000 7ff208eef700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff1e80682f0 0x7ff1e806a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009bc0 tx=0x7ff1f80091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.360+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff1f0053420 con 0x7ff2040cd8c0 2023-11-28T11:51:51.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.540+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff204143b60 con 0x7ff2040cd8c0 2023-11-28T11:51:51.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.540+0000 7ff2017fa700 1 -- 172.21.15.144:0/675703697 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff204143b60 con 0x7ff2040cd8c0 2023-11-28T11:51:51.543 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:51.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff1e80682f0 msgr2=0x7ff1e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff1e80682f0 0x7ff1e806a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009bc0 tx=0x7ff1f80091c0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 msgr2=0x7ff2040c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 0x7ff2040c6270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009b60 tx=0x7ff1f0004750 comp rx=0 tx=0).stop 2023-11-28T11:51:51.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 shutdown_connections 2023-11-28T11:51:51.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff1e80682f0 0x7ff1e806a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2040cdc90 0x7ff2040c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2040b2e70 0x7ff2041f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 --2- 172.21.15.144:0/675703697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2040cd8c0 0x7ff2040c6270 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 >> 172.21.15.144:0/675703697 conn(0x7ff20406aed0 msgr2=0x7ff20413f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:51.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 shutdown_connections 2023-11-28T11:51:51.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.543+0000 7ff20b153700 1 -- 172.21.15.144:0/675703697 wait complete. 2023-11-28T11:51:51.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:51:51.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.677+0000 7fe9984f4700 1 Processor -- start 2023-11-28T11:51:51.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.677+0000 7fe9984f4700 1 -- start start 2023-11-28T11:51:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe9984f4700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe990071040 0x7fe990071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe9984f4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe99006dee0 0x7fe99006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe9984f4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe990071950 con 0x7fe99006cdc0 2023-11-28T11:51:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe9984f4700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe990161550 con 0x7fe990071040 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe9984f4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99015d060 con 0x7fe99006dee0 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe990071040 0x7fe990071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe990071040 0x7fe990071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35690/0 (socket says 172.21.15.144:35690) 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 -- 172.21.15.144:0/420676037 learned_addr learned my addr 172.21.15.144:0/420676037 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe995a8f700 1 --2- 172.21.15.144:0/420676037 >> v2:172.21.15.177:3300/0 conn(0x7fe99006dee0 0x7fe99006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 -- 172.21.15.144:0/420676037 >> v2:172.21.15.177:3300/0 conn(0x7fe99006dee0 msgr2=0x7fe99006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 --2- 172.21.15.144:0/420676037 >> v2:172.21.15.177:3300/0 conn(0x7fe99006dee0 0x7fe99006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 -- 172.21.15.144:0/420676037 >> v1:172.21.15.144:6789/0 conn(0x7fe99006cdc0 legacy=0x7fe99006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe996290700 1 -- 172.21.15.144:0/420676037 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99015d340 con 0x7fe990071040 2023-11-28T11:51:51.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.678+0000 7fe99528e700 1 -- 172.21.15.144:0/420676037 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007899020 0 0) 0x7fe990071950 con 0x7fe99006cdc0 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe996290700 1 --2- 172.21.15.144:0/420676037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe990071410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe99006fb00 tx=0x7fe980009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd128307c082762 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe99528e700 1 -- 172.21.15.144:0/420676037 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000b870 con 0x7fe990071040 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe99528e700 1 -- 172.21.15.144:0/420676037 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98000b9d0 con 0x7fe990071040 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe99528e700 1 -- 172.21.15.144:0/420676037 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000bd60 con 0x7fe990071040 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 -- 172.21.15.144:0/420676037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 msgr2=0x7fe990071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 --2- 172.21.15.144:0/420676037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe990071410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe99006fb00 tx=0x7fe980009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 -- 172.21.15.144:0/420676037 shutdown_connections 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 --2- 172.21.15.144:0/420676037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe990071410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 --2- 172.21.15.144:0/420676037 >> v2:172.21.15.177:3300/0 conn(0x7fe99006dee0 0x7fe99006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 -- 172.21.15.144:0/420676037 >> 172.21.15.144:0/420676037 conn(0x7fe9900d0000 msgr2=0x7fe9900d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 -- 172.21.15.144:0/420676037 shutdown_connections 2023-11-28T11:51:51.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.679+0000 7fe9984f4700 1 -- 172.21.15.144:0/420676037 wait complete. 2023-11-28T11:51:51.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.680+0000 7fe9984f4700 1 Processor -- start 2023-11-28T11:51:51.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.680+0000 7fe9984f4700 1 -- start start 2023-11-28T11:51:51.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe9984f4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe99006cdc0 0x7fe9901e8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe9984f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe99006dee0 0x7fe9901577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe9984f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe9900ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe9984f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990152b50 con 0x7fe990071040 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe9984f4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990157d00 con 0x7fe99006cdc0 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe9984f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901e7aa0 con 0x7fe99006dee0 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe996a91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe9900ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe996a91700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe9900ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35700/0 (socket says 172.21.15.144:35700) 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe996a91700 1 -- 172.21.15.144:0/403463796 learned_addr learned my addr 172.21.15.144:0/403463796 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe995a8f700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe99006dee0 0x7fe9901577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.681+0000 7fe996290700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe99006cdc0 0x7fe9901e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe996a91700 1 -- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe99006dee0 msgr2=0x7fe9901577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe996a91700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe99006dee0 0x7fe9901577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe996a91700 1 -- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe99006cdc0 msgr2=0x7fe9901e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe996a91700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe99006cdc0 0x7fe9901e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe996a91700 1 -- 172.21.15.144:0/403463796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980009550 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe996a91700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe9900ade40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c7a0 tx=0x7fe98800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988003e30 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe988005070 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99015d900 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988005420 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.682+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99015d620 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.683+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99006e870 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.684+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe988019030 con 0x7fe990071040 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.684+0000 7fe986ffd700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9740682f0 0x7fe97406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:51.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.684+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe988085100 con 0x7fe990071040 2023-11-28T11:51:51.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.684+0000 7fe996290700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9740682f0 0x7fe97406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:51.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.684+0000 7fe996290700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9740682f0 0x7fe97406a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe980014040 tx=0x7fe980008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:51.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.688+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe988051960 con 0x7fe990071040 2023-11-28T11:51:51.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.887+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9900720a0 con 0x7fe990071040 2023-11-28T11:51:51.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.887+0000 7fe986ffd700 1 -- 172.21.15.144:0/403463796 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe988017080 con 0x7fe990071040 2023-11-28T11:51:51.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.889+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9740682f0 msgr2=0x7fe97406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.889+0000 7fe9984f4700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9740682f0 0x7fe97406a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe980014040 tx=0x7fe980008030 comp rx=0 tx=0).stop 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.889+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 msgr2=0x7fe9900ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.889+0000 7fe9984f4700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe9900ade40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c7a0 tx=0x7fe98800cab0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 shutdown_connections 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe990071040 0x7fe9900ade40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9740682f0 0x7fe97406a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe99006cdc0 0x7fe9901e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 --2- 172.21.15.144:0/403463796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe99006dee0 0x7fe9901577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 >> 172.21.15.144:0/403463796 conn(0x7fe9900d0000 msgr2=0x7fe99006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 shutdown_connections 2023-11-28T11:51:51.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:51.890+0000 7fe9984f4700 1 -- 172.21.15.144:0/403463796 wait complete. 2023-11-28T11:51:51.893 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:51:51.903 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T11:51:51.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:51.904 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T11:51:51.941 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:51:51.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:51.941 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:51:51.998 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:51.999 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:52.020 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:52.020 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:52.081 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:51:52.081 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:51:52.142 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:52.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.260+0000 7f758e8d0700 1 Processor -- start 2023-11-28T11:51:52.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.260+0000 7f758e8d0700 1 -- start start 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f758e8d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 0x7f75880b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f758e8d0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758813ff00 con 0x7f75880b2e70 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f758e8d0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75881404c0 con 0x7f75880cd8c0 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f758e8d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7588140dc0 con 0x7f7588140950 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f758ce6d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f75880b2e70 0x7f758806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52420/0 (socket says 172.21.15.144:52420) 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f758ce6d700 1 -- 172.21.15.144:0/3852271532 learned_addr learned my addr 172.21.15.144:0/3852271532 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738598984 0 0) 0x7f758813ff00 con 0x7f75880b2e70 2023-11-28T11:51:52.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75700035b0 con 0x7f75880b2e70 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f75877fe700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 0x7f75880b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 212342120 0 0) 0x7f75700035b0 con 0x7f75880b2e70 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 msgr2=0x7f75880b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 0x7f75880b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 >> v1:172.21.15.177:6789/0 conn(0x7f75880cd8c0 legacy=0x7f75880cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758815aa80 con 0x7f75880b2e70 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f75800019f0 con 0x7f75880b2e70 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 >> v1:172.21.15.144:6789/0 conn(0x7f75880b2e70 legacy=0x7f758806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.261+0000 7f7586ffd700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 0x7f7570003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f7570004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7570004d90 0x7f7570005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7588140dc0 con 0x7f7570004060 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757000bc20 con 0x7f7570004d90 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757000bde0 con 0x7f7588140950 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f7570004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7587fff700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 0x7f7570003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f758ce6d700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7570004d90 0x7f7570005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 -- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 msgr2=0x7f7570003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588140950 0x7f7570003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 -- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7570004d90 msgr2=0x7f7570005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7570004d90 0x7f7570005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 -- 172.21.15.144:0/3852271532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700035b0 con 0x7f7570004060 2023-11-28T11:51:52.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f75877fe700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f7570004850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7578009860 tx=0x7f7570005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d591e3ca63bfe398 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757800c070 con 0x7f7570004060 2023-11-28T11:51:52.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7578004020 con 0x7f7570004060 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.262+0000 7f7586ffd700 1 -- 172.21.15.144:0/3852271532 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780043b0 con 0x7f7570004060 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758e8d0700 1 -- 172.21.15.144:0/3852271532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 msgr2=0x7f7570004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758e8d0700 1 --2- 172.21.15.144:0/3852271532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f7570004850 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7578009860 tx=0x7f7570005880 comp rx=0 tx=0).stop 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758ce6d700 1 -- 172.21.15.144:0/3852271532 reap_dead start 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758e8d0700 1 -- 172.21.15.144:0/3852271532 shutdown_connections 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758e8d0700 1 -- 172.21.15.144:0/3852271532 >> 172.21.15.144:0/3852271532 conn(0x7f758806aed0 msgr2=0x7f758806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:52.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758e8d0700 1 -- 172.21.15.144:0/3852271532 shutdown_connections 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.263+0000 7f758e8d0700 1 -- 172.21.15.144:0/3852271532 wait complete. 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.264+0000 7f758e8d0700 1 Processor -- start 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.264+0000 7f758e8d0700 1 -- start start 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f758e8d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f758813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f758e8d0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f758806b380 0x7f758813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f758e8d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588143170 0x7f7588142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f758e8d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c7f80 con 0x7f7570004060 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f758e8d0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75881f23d0 con 0x7f758806b380 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f758e8d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758806c370 con 0x7f7588143170 2023-11-28T11:51:52.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f758813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f758813ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35716/0 (socket says 172.21.15.144:35716) 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 -- 172.21.15.144:0/2308054603 learned_addr learned my addr 172.21.15.144:0/2308054603 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 -- 172.21.15.144:0/2308054603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588143170 msgr2=0x7f7588142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588143170 0x7f7588142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 -- 172.21.15.144:0/2308054603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f758806b380 msgr2=0x7f758813d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f758806b380 0x7f758813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 -- 172.21.15.144:0/2308054603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880b35f0 con 0x7f7570004060 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.265+0000 7f7587fff700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f758813ccf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f757c00f770 tx=0x7f757c00fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.266+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c0014f0 con 0x7f7570004060 2023-11-28T11:51:52.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.266+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c0052e0 con 0x7f7570004060 2023-11-28T11:51:52.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.266+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c0036d0 con 0x7f7570004060 2023-11-28T11:51:52.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.266+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75880b3310 con 0x7f7570004060 2023-11-28T11:51:52.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.266+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588140150 con 0x7f7570004060 2023-11-28T11:51:52.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.267+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758804e620 con 0x7f7570004060 2023-11-28T11:51:52.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.267+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f757c003830 con 0x7f7570004060 2023-11-28T11:51:52.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.267+0000 7f7584ff9700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75740682f0 0x7f757406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.267+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f757c085c60 con 0x7f7570004060 2023-11-28T11:51:52.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.267+0000 7f75877fe700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75740682f0 0x7f757406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.268+0000 7f75877fe700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75740682f0 0x7f757406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75880cce60 tx=0x7f7578013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.271+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f757c0524c0 con 0x7f7570004060 2023-11-28T11:51:52.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.452+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75880c6500 con 0x7f7570004060 2023-11-28T11:51:52.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.453+0000 7f7584ff9700 1 -- 172.21.15.144:0/2308054603 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f757c055ae0 con 0x7f7570004060 2023-11-28T11:51:52.455 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:52.456 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:52.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75740682f0 msgr2=0x7f757406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.458 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75740682f0 0x7f757406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75880cce60 tx=0x7f7578013000 comp rx=0 tx=0).stop 2023-11-28T11:51:52.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 msgr2=0x7f758813ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f758813ccf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f757c00f770 tx=0x7f757c00fb30 comp rx=0 tx=0).stop 2023-11-28T11:51:52.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 shutdown_connections 2023-11-28T11:51:52.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7570004060 0x7f758813ccf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75740682f0 0x7f757406a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7588143170 0x7f7588142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 --2- 172.21.15.144:0/2308054603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f758806b380 0x7f758813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.455+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 >> 172.21.15.144:0/2308054603 conn(0x7f758806aed0 msgr2=0x7f75880cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:52.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.456+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 shutdown_connections 2023-11-28T11:51:52.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.456+0000 7f758e8d0700 1 -- 172.21.15.144:0/2308054603 wait complete. 2023-11-28T11:51:52.460 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:51:52.462 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:51:52.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:52.470 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:52.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.632+0000 7fd93edfe700 1 Processor -- start 2023-11-28T11:51:52.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.632+0000 7fd93edfe700 1 -- start start 2023-11-28T11:51:52.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd93edfe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 0x7fd9380cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd93edfe700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93806bbf0 con 0x7fd93806b060 2023-11-28T11:51:52.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd93edfe700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9380cc030 con 0x7fd938160b10 2023-11-28T11:51:52.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd93edfe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ca720 con 0x7fd938160740 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd93d39b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd93806b060 0x7fd93806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52440/0 (socket says 172.21.15.144:52440) 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd93d39b700 1 -- 172.21.15.144:0/3000968046 learned_addr learned my addr 172.21.15.144:0/3000968046 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.633+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237740849 0 0) 0x7fd93806bbf0 con 0x7fd93806b060 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd93cb9a700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 0x7fd9380cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9280035b0 con 0x7fd93806b060 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 906246417 0 0) 0x7fd9280035b0 con 0x7fd93806b060 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 msgr2=0x7fd9380cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 0x7fd9380cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 >> v1:172.21.15.177:6789/0 conn(0x7fd938160b10 legacy=0x7fd9380cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380c9e10 con 0x7fd93806b060 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fd9300019f0 con 0x7fd93806b060 2023-11-28T11:51:52.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 >> v1:172.21.15.144:6789/0 conn(0x7fd93806b060 legacy=0x7fd93806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 0x7fd928003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd928003f70 0x7fd928004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928004d90 0x7fd928005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92800ba10 con 0x7fd928003f70 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92800bb80 con 0x7fd928004d90 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92800bd40 con 0x7fd938160740 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd928003f70 0x7fd928004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd93cb9a700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 0x7fd928003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd93d39b700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928004d90 0x7fd928005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 -- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 msgr2=0x7fd928003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938160740 0x7fd928003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.639 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 -- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928004d90 msgr2=0x7fd928005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928004d90 0x7fd928005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 -- 172.21.15.144:0/3000968046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280035b0 con 0x7fd928003f70 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.634+0000 7fd937fff700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd928003f70 0x7fd928004850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd9240017c0 tx=0x7fd928005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cceae42f6ed0458 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9240052d0 con 0x7fd928003f70 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924005430 con 0x7fd928003f70 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd9377fe700 1 -- 172.21.15.144:0/3000968046 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9240038c0 con 0x7fd928003f70 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93edfe700 1 -- 172.21.15.144:0/3000968046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd928003f70 msgr2=0x7fd928004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93edfe700 1 --2- 172.21.15.144:0/3000968046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd928003f70 0x7fd928004850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd9240017c0 tx=0x7fd928005880 comp rx=0 tx=0).stop 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93d39b700 1 -- 172.21.15.144:0/3000968046 reap_dead start 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93edfe700 1 -- 172.21.15.144:0/3000968046 shutdown_connections 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93edfe700 1 -- 172.21.15.144:0/3000968046 >> 172.21.15.144:0/3000968046 conn(0x7fd9380b12b0 msgr2=0x7fd9380b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:52.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93edfe700 1 -- 172.21.15.144:0/3000968046 shutdown_connections 2023-11-28T11:51:52.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.635+0000 7fd93edfe700 1 -- 172.21.15.144:0/3000968046 wait complete. 2023-11-28T11:51:52.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.636+0000 7fd93edfe700 1 Processor -- start 2023-11-28T11:51:52.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.636+0000 7fd93edfe700 1 -- start start 2023-11-28T11:51:52.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93edfe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928003f70 0x7fd938159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93edfe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938157e90 0x7fd9381540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93edfe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 0x7fd9380ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93edfe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380ae870 con 0x7fd9380adeb0 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93edfe700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd938158ef0 con 0x7fd928003f70 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93edfe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9381595a0 con 0x7fd938157e90 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93d39b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 0x7fd9380ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93d39b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 0x7fd9380ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35740/0 (socket says 172.21.15.144:35740) 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.637+0000 7fd93d39b700 1 -- 172.21.15.144:0/2159492467 learned_addr learned my addr 172.21.15.144:0/2159492467 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd937fff700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938157e90 0x7fd9381540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93cb9a700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928003f70 0x7fd938159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93d39b700 1 -- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938157e90 msgr2=0x7fd9381540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93d39b700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938157e90 0x7fd9381540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93d39b700 1 -- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928003f70 msgr2=0x7fd938159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93d39b700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928003f70 0x7fd938159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93d39b700 1 -- 172.21.15.144:0/2159492467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380ca0d0 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd937fff700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938157e90 0x7fd9381540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93cb9a700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928003f70 0x7fd938159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93d39b700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 0x7fd9380ae330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd930002230 tx=0x7fd930005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930008770 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9300088f0 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93806efc0 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930015450 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.638+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9380ca3b0 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.639+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd938066110 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.639+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd93001c030 con 0x7fd9380adeb0 2023-11-28T11:51:52.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.640+0000 7fd9357fa700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd91c0682f0 0x7fd91c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.640+0000 7fd93cb9a700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd91c0682f0 0x7fd91c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.640+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd930085cd0 con 0x7fd9380adeb0 2023-11-28T11:51:52.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.640+0000 7fd93cb9a700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd91c0682f0 0x7fd91c06a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009ae0 tx=0x7fd92c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.644+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd9300524d0 con 0x7fd9380adeb0 2023-11-28T11:51:52.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.826+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd9380b5740 con 0x7fd9380adeb0 2023-11-28T11:51:52.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.826+0000 7fd9357fa700 1 -- 172.21.15.144:0/2159492467 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd93001a070 con 0x7fd9380adeb0 2023-11-28T11:51:52.828 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:52.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd91c0682f0 msgr2=0x7fd91c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd91c0682f0 0x7fd91c06a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009ae0 tx=0x7fd92c008040 comp rx=0 tx=0).stop 2023-11-28T11:51:52.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 msgr2=0x7fd9380ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 0x7fd9380ae330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd930002230 tx=0x7fd930005e90 comp rx=0 tx=0).stop 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 shutdown_connections 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9380adeb0 0x7fd9380ae330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd91c0682f0 0x7fd91c06a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd928003f70 0x7fd938159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 --2- 172.21.15.144:0/2159492467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd938157e90 0x7fd9381540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 >> 172.21.15.144:0/2159492467 conn(0x7fd9380b12b0 msgr2=0x7fd9380afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 shutdown_connections 2023-11-28T11:51:52.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.828+0000 7fd93edfe700 1 -- 172.21.15.144:0/2159492467 wait complete. 2023-11-28T11:51:52.842 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:52.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.966+0000 7f1fcd643700 1 Processor -- start 2023-11-28T11:51:52.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.966+0000 7f1fcd643700 1 -- start start 2023-11-28T11:51:52.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fcd643700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fc815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fcd643700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1fc80d0a00 0x7f1fc806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fcd643700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc806b420 con 0x7f1fc80b2930 2023-11-28T11:51:52.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fcd643700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc806b700 con 0x7f1fc80d0a00 2023-11-28T11:51:52.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fcd643700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc806b9e0 con 0x7f1fc80ce040 2023-11-28T11:51:52.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fc77fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1fc80b2930 0x7f1fc806c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52460/0 (socket says 172.21.15.144:52460) 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fc77fe700 1 -- 172.21.15.144:0/3598268804 learned_addr learned my addr 172.21.15.144:0/3598268804 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119629302 0 0) 0x7f1fc806b420 con 0x7f1fc80b2930 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb00035b0 con 0x7f1fc80b2930 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fc6ffd700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fc815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.967+0000 7f1fc67fc700 1 --2- 172.21.15.144:0/3598268804 >> v2:172.21.15.177:3300/0 conn(0x7f1fc80d0a00 0x7f1fc806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22722443 0 0) 0x7f1fb00035b0 con 0x7f1fc80b2930 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 msgr2=0x7f1fc815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fc815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 >> v2:172.21.15.177:3300/0 conn(0x7f1fc80d0a00 msgr2=0x7f1fc806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 --2- 172.21.15.144:0/3598268804 >> v2:172.21.15.177:3300/0 conn(0x7f1fc80d0a00 0x7f1fc806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc806bcc0 con 0x7f1fc80b2930 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f1fc00019f0 con 0x7f1fc80b2930 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 >> v1:172.21.15.144:6789/0 conn(0x7f1fc80b2930 legacy=0x7f1fc806c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fb0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb0003f80 0x7f1fb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0004d90 0x7f1fb0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc806b700 con 0x7f1fb0003f80 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc806b9e0 con 0x7f1fb0004d90 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb000ba10 con 0x7f1fc80ce040 2023-11-28T11:51:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb0003f80 0x7f1fb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc6ffd700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fb0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc77fe700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0004d90 0x7f1fb0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 -- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 msgr2=0x7f1fb0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fb0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 -- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0004d90 msgr2=0x7f1fb0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0004d90 0x7f1fb0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 -- 172.21.15.144:0/3598268804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00035b0 con 0x7f1fb0003f80 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc6ffd700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc80ce040 0x7f1fb0003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc67fc700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb0003f80 0x7f1fb0004850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009b60 tx=0x7f1fb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e071a89a0a2542e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800e6c0 con 0x7f1fb0003f80 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb800e820 con 0x7f1fb0003f80 2023-11-28T11:51:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.968+0000 7f1fc5ffb700 1 -- 172.21.15.144:0/3598268804 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800eb90 con 0x7f1fb0003f80 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fcd643700 1 -- 172.21.15.144:0/3598268804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb0003f80 msgr2=0x7f1fb0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fcd643700 1 --2- 172.21.15.144:0/3598268804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fb0003f80 0x7f1fb0004850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009b60 tx=0x7f1fb0005880 comp rx=0 tx=0).stop 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fc77fe700 1 -- 172.21.15.144:0/3598268804 reap_dead start 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fcd643700 1 -- 172.21.15.144:0/3598268804 shutdown_connections 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fcd643700 1 -- 172.21.15.144:0/3598268804 >> 172.21.15.144:0/3598268804 conn(0x7f1fc806aed0 msgr2=0x7f1fc806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fcd643700 1 -- 172.21.15.144:0/3598268804 shutdown_connections 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.969+0000 7f1fcd643700 1 -- 172.21.15.144:0/3598268804 wait complete. 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.970+0000 7f1fcd643700 1 Processor -- start 2023-11-28T11:51:52.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.970+0000 7f1fcd643700 1 -- start start 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fcd643700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0003f80 0x7f1fc80c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fcd643700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc81f1800 0x7f1fc81f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fcd643700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 0x7f1fc80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fcd643700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80b6c50 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fcd643700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc81f1c70 con 0x7f1fb0003f80 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fcd643700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80b6690 con 0x7f1fc81f1800 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fc77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 0x7f1fc80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fc77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 0x7f1fc80cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35760/0 (socket says 172.21.15.144:35760) 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fc77fe700 1 -- 172.21.15.144:0/4226968465 learned_addr learned my addr 172.21.15.144:0/4226968465 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fc6ffd700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0003f80 0x7f1fc80c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.971+0000 7f1fc67fc700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc81f1800 0x7f1fc81f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fc77fe700 1 -- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc81f1800 msgr2=0x7f1fc81f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fc77fe700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc81f1800 0x7f1fc81f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fc77fe700 1 -- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0003f80 msgr2=0x7f1fc80c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fc77fe700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0003f80 0x7f1fc80c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fc77fe700 1 -- 172.21.15.144:0/4226968465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc806fd90 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fc77fe700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 0x7f1fc80cbf90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80b3980 tx=0x7f1fc0007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000eb40 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc000eca0 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0013420 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc806bfa0 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.972+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc806ff20 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.973+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc8068690 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.973+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1fc0013580 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.974+0000 7f1faf7fe700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fb4068320 0x7f1fb406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.974+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fc0086230 con 0x7f1fc81f5cc0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.974+0000 7f1fc6ffd700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fb4068320 0x7f1fb406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.974+0000 7f1fc6ffd700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fb4068320 0x7f1fb406a7d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009af0 tx=0x7f1fbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:52.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:52.978+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1fc0052a90 con 0x7f1fc81f5cc0 2023-11-28T11:51:53.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.158+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fc80d1a30 con 0x7f1fc81f5cc0 2023-11-28T11:51:53.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.158+0000 7f1faf7fe700 1 -- 172.21.15.144:0/4226968465 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fc00560b0 con 0x7f1fc81f5cc0 2023-11-28T11:51:53.160 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:53.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fb4068320 msgr2=0x7f1fb406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fb4068320 0x7f1fb406a7d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009af0 tx=0x7f1fbc008040 comp rx=0 tx=0).stop 2023-11-28T11:51:53.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 msgr2=0x7f1fc80cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 0x7f1fc80cbf90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80b3980 tx=0x7f1fc0007c90 comp rx=0 tx=0).stop 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 shutdown_connections 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc81f5cc0 0x7f1fc80cbf90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1fb4068320 0x7f1fb406a7d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fb0003f80 0x7f1fc80c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 --2- 172.21.15.144:0/4226968465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1fc81f1800 0x7f1fc81f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 >> 172.21.15.144:0/4226968465 conn(0x7f1fc806aed0 msgr2=0x7f1fc815ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 shutdown_connections 2023-11-28T11:51:53.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.160+0000 7f1fcd643700 1 -- 172.21.15.144:0/4226968465 wait complete. 2023-11-28T11:51:53.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:53.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.296+0000 7f7d1a537700 1 Processor -- start 2023-11-28T11:51:53.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.296+0000 7f7d1a537700 1 -- start start 2023-11-28T11:51:53.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d1a537700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7d14071040 0x7f7d14071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d1a537700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7d1406dee0 0x7f7d1406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d1a537700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d14071950 con 0x7f7d1406cdc0 2023-11-28T11:51:53.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d1a537700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d14161550 con 0x7f7d14071040 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d1a537700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1415d060 con 0x7f7d1406dee0 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7d14071040 0x7f7d14071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7d14071040 0x7f7d14071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35764/0 (socket says 172.21.15.144:35764) 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 -- 172.21.15.144:0/1786877729 learned_addr learned my addr 172.21.15.144:0/1786877729 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d137fe700 1 --2- 172.21.15.144:0/1786877729 >> v2:172.21.15.177:3300/0 conn(0x7f7d1406dee0 0x7f7d1406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 -- 172.21.15.144:0/1786877729 >> v2:172.21.15.177:3300/0 conn(0x7f7d1406dee0 msgr2=0x7f7d1406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 --2- 172.21.15.144:0/1786877729 >> v2:172.21.15.177:3300/0 conn(0x7f7d1406dee0 0x7f7d1406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 -- 172.21.15.144:0/1786877729 >> v1:172.21.15.144:6789/0 conn(0x7f7d1406cdc0 legacy=0x7f7d1406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 -- 172.21.15.144:0/1786877729 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1415d340 con 0x7f7d14071040 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d12ffd700 1 -- 172.21.15.144:0/1786877729 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416655327 0 0) 0x7f7d14071950 con 0x7f7d1406cdc0 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d13fff700 1 --2- 172.21.15.144:0/1786877729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d14071410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d1406fb00 tx=0x7f7d04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d7a6c228ae3475 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d12ffd700 1 -- 172.21.15.144:0/1786877729 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04011830 con 0x7f7d14071040 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d12ffd700 1 -- 172.21.15.144:0/1786877729 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04011990 con 0x7f7d14071040 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.297+0000 7f7d12ffd700 1 -- 172.21.15.144:0/1786877729 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04011d20 con 0x7f7d14071040 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 -- 172.21.15.144:0/1786877729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 msgr2=0x7f7d14071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 --2- 172.21.15.144:0/1786877729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d14071410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d1406fb00 tx=0x7f7d04009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 -- 172.21.15.144:0/1786877729 shutdown_connections 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 --2- 172.21.15.144:0/1786877729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d14071410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 --2- 172.21.15.144:0/1786877729 >> v2:172.21.15.177:3300/0 conn(0x7f7d1406dee0 0x7f7d1406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 -- 172.21.15.144:0/1786877729 >> 172.21.15.144:0/1786877729 conn(0x7f7d140d0000 msgr2=0x7f7d140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:53.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 -- 172.21.15.144:0/1786877729 shutdown_connections 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.298+0000 7f7d1a537700 1 -- 172.21.15.144:0/1786877729 wait complete. 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.299+0000 7f7d1a537700 1 Processor -- start 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.300+0000 7f7d1a537700 1 -- start start 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d1406cdc0 0x7f7d141e8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d1406dee0 0x7f7d141577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d140ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14152b50 con 0x7f7d14071040 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14157d00 con 0x7f7d1406dee0 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d141e7aa0 con 0x7f7d1406cdc0 2023-11-28T11:51:53.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d140ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d140ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35774/0 (socket says 172.21.15.144:35774) 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 -- 172.21.15.144:0/3895362358 learned_addr learned my addr 172.21.15.144:0/3895362358 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d137fe700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d1406dee0 0x7f7d141577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d13fff700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d1406cdc0 0x7f7d141e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 -- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d1406cdc0 msgr2=0x7f7d141e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d1406cdc0 0x7f7d141e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 -- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d1406dee0 msgr2=0x7f7d141577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d1406dee0 0x7f7d141577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 -- 172.21.15.144:0/3895362358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04009550 con 0x7f7d14071040 2023-11-28T11:51:53.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d18ad4700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d140ade40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00c7a0 tx=0x7f7d0c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c003e30 con 0x7f7d14071040 2023-11-28T11:51:53.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c005070 con 0x7f7d14071040 2023-11-28T11:51:53.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1415d900 con 0x7f7d14071040 2023-11-28T11:51:53.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005420 con 0x7f7d14071040 2023-11-28T11:51:53.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.301+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1415d620 con 0x7f7d14071040 2023-11-28T11:51:53.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.302+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1406e870 con 0x7f7d14071040 2023-11-28T11:51:53.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.303+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d0c019030 con 0x7f7d14071040 2023-11-28T11:51:53.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.303+0000 7f7d10ff9700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7d000682f0 0x7f7d0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.303+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d0c085170 con 0x7f7d14071040 2023-11-28T11:51:53.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.303+0000 7f7d13fff700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7d000682f0 0x7f7d0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.304+0000 7f7d13fff700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7d000682f0 0x7f7d0006a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d04014040 tx=0x7f7d04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.307+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d0c051950 con 0x7f7d14071040 2023-11-28T11:51:53.491 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.489+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d140720a0 con 0x7f7d14071040 2023-11-28T11:51:53.491 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.490+0000 7f7d10ff9700 1 -- 172.21.15.144:0/3895362358 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d0c054f70 con 0x7f7d14071040 2023-11-28T11:51:53.491 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.491+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7d000682f0 msgr2=0x7f7d0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.491+0000 7f7d1a537700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7d000682f0 0x7f7d0006a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d04014040 tx=0x7f7d04008030 comp rx=0 tx=0).stop 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 msgr2=0x7f7d140ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d140ade40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00c7a0 tx=0x7f7d0c00cab0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 shutdown_connections 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d14071040 0x7f7d140ade40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7d000682f0 0x7f7d0006a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7d1406cdc0 0x7f7d141e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 --2- 172.21.15.144:0/3895362358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d1406dee0 0x7f7d141577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 >> 172.21.15.144:0/3895362358 conn(0x7f7d140d0000 msgr2=0x7f7d1406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 shutdown_connections 2023-11-28T11:51:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.492+0000 7f7d1a537700 1 -- 172.21.15.144:0/3895362358 wait complete. 2023-11-28T11:51:53.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:53.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.627+0000 7f0c47628700 1 Processor -- start 2023-11-28T11:51:53.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.627+0000 7f0c47628700 1 -- start start 2023-11-28T11:51:53.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c47628700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c400cd8c0 0x7f0c40140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c47628700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c400cdc90 0x7f0c400b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c47628700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4013ff00 con 0x7f0c400b2e70 2023-11-28T11:51:53.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c47628700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c401404c0 con 0x7f0c400cdc90 2023-11-28T11:51:53.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c47628700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4015aa80 con 0x7f0c400cd8c0 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c400cdc90 0x7f0c400b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c400cdc90 0x7f0c400b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35780/0 (socket says 172.21.15.144:35780) 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 -- 172.21.15.144:0/1096892274 learned_addr learned my addr 172.21.15.144:0/1096892274 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c453c4700 1 --2- 172.21.15.144:0/1096892274 >> v2:172.21.15.177:3300/0 conn(0x7f0c400cd8c0 0x7f0c40140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 -- 172.21.15.144:0/1096892274 >> v2:172.21.15.177:3300/0 conn(0x7f0c400cd8c0 msgr2=0x7f0c40140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 --2- 172.21.15.144:0/1096892274 >> v2:172.21.15.177:3300/0 conn(0x7f0c400cd8c0 0x7f0c40140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 -- 172.21.15.144:0/1096892274 >> v1:172.21.15.144:6789/0 conn(0x7f0c400b2e70 legacy=0x7f0c4006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c37fff700 1 -- 172.21.15.144:0/1096892274 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775834522 0 0) 0x7f0c4013ff00 con 0x7f0c400b2e70 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.628+0000 7f0c44bc3700 1 -- 172.21.15.144:0/1096892274 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400b3310 con 0x7f0c400cdc90 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c44bc3700 1 --2- 172.21.15.144:0/1096892274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400cdc90 0x7f0c400b2930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c40070a20 tx=0x7f0c30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93afd11f28d7b2fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c37fff700 1 -- 172.21.15.144:0/1096892274 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011830 con 0x7f0c400cdc90 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c37fff700 1 -- 172.21.15.144:0/1096892274 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30011990 con 0x7f0c400cdc90 2023-11-28T11:51:53.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c37fff700 1 -- 172.21.15.144:0/1096892274 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011d20 con 0x7f0c400cdc90 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 -- 172.21.15.144:0/1096892274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400cdc90 msgr2=0x7f0c400b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 --2- 172.21.15.144:0/1096892274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400cdc90 0x7f0c400b2930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0c40070a20 tx=0x7f0c30009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 -- 172.21.15.144:0/1096892274 shutdown_connections 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 --2- 172.21.15.144:0/1096892274 >> v2:172.21.15.177:3300/0 conn(0x7f0c400cd8c0 0x7f0c40140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 --2- 172.21.15.144:0/1096892274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400cdc90 0x7f0c400b2930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 -- 172.21.15.144:0/1096892274 >> 172.21.15.144:0/1096892274 conn(0x7f0c4006aed0 msgr2=0x7f0c4006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 -- 172.21.15.144:0/1096892274 shutdown_connections 2023-11-28T11:51:53.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.629+0000 7f0c47628700 1 -- 172.21.15.144:0/1096892274 wait complete. 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.630+0000 7f0c47628700 1 Processor -- start 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 -- start start 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 0x7f0c401f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c400cd8c0 0x7f0c400c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c400cdc90 0x7f0c400c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40143080 con 0x7f0c400b2e70 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40143360 con 0x7f0c400cd8c0 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c47628700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40143640 con 0x7f0c400cdc90 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c453c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 0x7f0c401f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c453c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 0x7f0c401f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35784/0 (socket says 172.21.15.144:35784) 2023-11-28T11:51:53.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.631+0000 7f0c453c4700 1 -- 172.21.15.144:0/1139322027 learned_addr learned my addr 172.21.15.144:0/1139322027 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c44bc3700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c400cd8c0 0x7f0c400c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c45bc5700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c400cdc90 0x7f0c400c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c453c4700 1 -- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c400cdc90 msgr2=0x7f0c400c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c453c4700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c400cdc90 0x7f0c400c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c453c4700 1 -- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c400cd8c0 msgr2=0x7f0c400c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c453c4700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c400cd8c0 0x7f0c400c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c453c4700 1 -- 172.21.15.144:0/1139322027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c30009550 con 0x7f0c400b2e70 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c44bc3700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c400cd8c0 0x7f0c400c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c45bc5700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c400cdc90 0x7f0c400c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c453c4700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 0x7f0c401f0640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e990 tx=0x7f0c3c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0096f0 con 0x7f0c400b2e70 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c00c850 con 0x7f0c400b2e70 2023-11-28T11:51:53.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c40140150 con 0x7f0c400b2e70 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00cbc0 con 0x7f0c400b2e70 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.632+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c400b35f0 con 0x7f0c400b2e70 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.633+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c400ce100 con 0x7f0c400b2e70 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.633+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c3c023070 con 0x7f0c400b2e70 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.634+0000 7f0c35ffb700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c240682f0 0x7f0c2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.634+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c3c086010 con 0x7f0c400b2e70 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.634+0000 7f0c44bc3700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c240682f0 0x7f0c2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.634+0000 7f0c44bc3700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c240682f0 0x7f0c2406a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c40070a20 tx=0x7f0c30008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.638+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c3c052870 con 0x7f0c400b2e70 2023-11-28T11:51:53.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.823+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c40143c90 con 0x7f0c400b2e70 2023-11-28T11:51:53.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.823+0000 7f0c35ffb700 1 -- 172.21.15.144:0/1139322027 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c3c055e90 con 0x7f0c400b2e70 2023-11-28T11:51:53.825 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c240682f0 msgr2=0x7f0c2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c240682f0 0x7f0c2406a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c40070a20 tx=0x7f0c30008030 comp rx=0 tx=0).stop 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 msgr2=0x7f0c401f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 0x7f0c401f0640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00e990 tx=0x7f0c3c00ed50 comp rx=0 tx=0).stop 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 shutdown_connections 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c400b2e70 0x7f0c401f0640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c240682f0 0x7f0c2406a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c400cdc90 0x7f0c400c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 --2- 172.21.15.144:0/1139322027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c400cd8c0 0x7f0c400c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 >> 172.21.15.144:0/1139322027 conn(0x7f0c4006aed0 msgr2=0x7f0c4013f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 shutdown_connections 2023-11-28T11:51:53.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.826+0000 7f0c47628700 1 -- 172.21.15.144:0/1139322027 wait complete. 2023-11-28T11:51:53.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:51:53.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.959+0000 7fc202f36700 1 Processor -- start 2023-11-28T11:51:53.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.959+0000 7fc202f36700 1 -- start start 2023-11-28T11:51:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc202f36700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc1fc071040 0x7fc1fc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc202f36700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc1fc160540 0x7fc1fc160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc202f36700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc071950 con 0x7fc1fc06dee0 2023-11-28T11:51:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc202f36700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc15d060 con 0x7fc1fc071040 2023-11-28T11:51:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc202f36700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc15d340 con 0x7fc1fc160540 2023-11-28T11:51:53.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc1fc071040 0x7fc1fc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc1fc071040 0x7fc1fc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35792/0 (socket says 172.21.15.144:35792) 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 -- 172.21.15.144:0/1809449053 learned_addr learned my addr 172.21.15.144:0/1809449053 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 -- 172.21.15.144:0/1809449053 >> v2:172.21.15.177:3300/0 conn(0x7fc1fc160540 msgr2=0x7fc1fc160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 --2- 172.21.15.144:0/1809449053 >> v2:172.21.15.177:3300/0 conn(0x7fc1fc160540 0x7fc1fc160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 -- 172.21.15.144:0/1809449053 >> v1:172.21.15.177:6789/0 conn(0x7fc1fc06dee0 legacy=0x7fc1fc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 -- 172.21.15.144:0/1809449053 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc15d620 con 0x7fc1fc071040 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc200cd2700 1 --2- 172.21.15.144:0/1809449053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc071040 0x7fc1fc071410 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0c3a10 tx=0x7fc1f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dee2475e3ab994 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc1fb7fe700 1 -- 172.21.15.144:0/1809449053 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0004230 con 0x7fc1fc071040 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc1fb7fe700 1 -- 172.21.15.144:0/1809449053 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f0004390 con 0x7fc1fc071040 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.960+0000 7fc1fb7fe700 1 -- 172.21.15.144:0/1809449053 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00036a0 con 0x7fc1fc071040 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 -- 172.21.15.144:0/1809449053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc071040 msgr2=0x7fc1fc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 --2- 172.21.15.144:0/1809449053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc071040 0x7fc1fc071410 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0c3a10 tx=0x7fc1f0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 -- 172.21.15.144:0/1809449053 shutdown_connections 2023-11-28T11:51:53.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 --2- 172.21.15.144:0/1809449053 >> v2:172.21.15.177:3300/0 conn(0x7fc1fc160540 0x7fc1fc160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 --2- 172.21.15.144:0/1809449053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc071040 0x7fc1fc071410 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 -- 172.21.15.144:0/1809449053 >> 172.21.15.144:0/1809449053 conn(0x7fc1fc0d0000 msgr2=0x7fc1fc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:53.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 -- 172.21.15.144:0/1809449053 shutdown_connections 2023-11-28T11:51:53.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.961+0000 7fc202f36700 1 -- 172.21.15.144:0/1809449053 wait complete. 2023-11-28T11:51:53.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.962+0000 7fc202f36700 1 Processor -- start 2023-11-28T11:51:53.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.962+0000 7fc202f36700 1 -- start start 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc202f36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 0x7fc1fc14a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc202f36700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1fc071040 0x7fc1fc0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc202f36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1fc160540 0x7fc1fc0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc202f36700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b4500 con 0x7fc1fc06dee0 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc202f36700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc14a9d0 con 0x7fc1fc071040 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc202f36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b4e60 con 0x7fc1fc160540 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc200cd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 0x7fc1fc14a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc200cd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 0x7fc1fc14a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35798/0 (socket says 172.21.15.144:35798) 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.963+0000 7fc200cd2700 1 -- 172.21.15.144:0/2381748872 learned_addr learned my addr 172.21.15.144:0/2381748872 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc200cd2700 1 -- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1fc160540 msgr2=0x7fc1fc0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc200cd2700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1fc160540 0x7fc1fc0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc1fbfff700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1fc071040 0x7fc1fc0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc200cd2700 1 -- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1fc071040 msgr2=0x7fc1fc0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc200cd2700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1fc071040 0x7fc1fc0b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc200cd2700 1 -- 172.21.15.144:0/2381748872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0009550 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc1fbfff700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1fc071040 0x7fc1fc0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc200cd2700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 0x7fc1fc14a490 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009130 tx=0x7fc1f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0011630 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f0011790 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc15ccd0 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0011b40 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.964+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc15d900 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.965+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc06a270 con 0x7fc1fc06dee0 2023-11-28T11:51:53.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.965+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc1f0017020 con 0x7fc1fc06dee0 2023-11-28T11:51:53.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.965+0000 7fc1f97fa700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1e00682f0 0x7fc1e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:53.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.966+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc1f0087580 con 0x7fc1fc06dee0 2023-11-28T11:51:53.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.966+0000 7fc1fbfff700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1e00682f0 0x7fc1e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:53.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.966+0000 7fc1fbfff700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1e00682f0 0x7fc1e006a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8006fd0 tx=0x7fc1e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:53.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:53.970+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc1f0053d60 con 0x7fc1fc06dee0 2023-11-28T11:51:54.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.165+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc1fc0720a0 con 0x7fc1fc06dee0 2023-11-28T11:51:54.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.166+0000 7fc1f97fa700 1 -- 172.21.15.144:0/2381748872 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc1f0057380 con 0x7fc1fc06dee0 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1e00682f0 msgr2=0x7fc1e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1e00682f0 0x7fc1e006a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8006fd0 tx=0x7fc1e8008040 comp rx=0 tx=0).stop 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 msgr2=0x7fc1fc14a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 0x7fc1fc14a490 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009130 tx=0x7fc1f0004750 comp rx=0 tx=0).stop 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 shutdown_connections 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1fc06dee0 0x7fc1fc14a490 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc1e00682f0 0x7fc1e006a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1fc160540 0x7fc1fc0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 --2- 172.21.15.144:0/2381748872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1fc071040 0x7fc1fc0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 >> 172.21.15.144:0/2381748872 conn(0x7fc1fc0d0000 msgr2=0x7fc1fc15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 shutdown_connections 2023-11-28T11:51:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.168+0000 7fc202f36700 1 -- 172.21.15.144:0/2381748872 wait complete. 2023-11-28T11:51:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:51:54.181 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T11:51:54.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:54.182 DEBUG:teuthology.orchestra.run.smithi177:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T11:51:54.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:51:54.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:51:54.234 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:51:54.253 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:51:54.254 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:54.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.372+0000 7f6c10f6d700 1 Processor -- start 2023-11-28T11:51:54.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.372+0000 7f6c10f6d700 1 -- start start 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c10f6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 0x7f6c0c167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c10f6d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c15d060 con 0x7f6c0c071040 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c10f6d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c15d340 con 0x7f6c0c06dee0 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c10f6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c161550 con 0x7f6c0c1658c0 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0a59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6c0c071040 0x7f6c0c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52518/0 (socket says 172.21.15.144:52518) 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0a59c700 1 -- 172.21.15.144:0/2913352302 learned_addr learned my addr 172.21.15.144:0/2913352302 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:54.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0ad9d700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 0x7f6c0c167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150733602 0 0) 0x7f6c0c15d060 con 0x7f6c0c071040 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf40035b0 con 0x7f6c0c071040 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 816606610 0 0) 0x7f6bf40035b0 con 0x7f6c0c071040 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 msgr2=0x7f6c0c167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 0x7f6c0c167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 >> v1:172.21.15.177:6789/0 conn(0x7f6c0c06dee0 legacy=0x7f6c0c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c15d620 con 0x7f6c0c071040 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f6bfc001670 con 0x7f6c0c071040 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 >> v1:172.21.15.144:6789/0 conn(0x7f6c0c071040 legacy=0x7f6c0c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bf4003a30 0x7f6bf4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6bf4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.373+0000 7f6c0959a700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 0x7f6bf40052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c161550 con 0x7f6bf4004850 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf400bbd0 con 0x7f6bf4003a30 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf400bd90 con 0x7f6c0c1658c0 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6bf4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0a59c700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bf4003a30 0x7f6bf4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0ad9d700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 0x7f6bf40052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 -- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 msgr2=0x7f6bf40052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c1658c0 0x7f6bf40052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 -- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bf4003a30 msgr2=0x7f6bf4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bf4003a30 0x7f6bf4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 -- 172.21.15.144:0/2913352302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40035b0 con 0x7f6bf4004850 2023-11-28T11:51:54.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0a59c700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bf4003a30 0x7f6bf4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c09d9b700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6bf4004cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6c00007fd0 tx=0x7f6bf4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dae1bbecc87745e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000012f0 con 0x7f6bf4004850 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c000052f0 con 0x7f6bf4004850 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.374+0000 7f6c0959a700 1 -- 172.21.15.144:0/2913352302 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00006860 con 0x7f6bf4004850 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c10f6d700 1 -- 172.21.15.144:0/2913352302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 msgr2=0x7f6bf4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/2913352302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6bf4004cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6c00007fd0 tx=0x7f6bf4005880 comp rx=0 tx=0).stop 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c0ad9d700 1 -- 172.21.15.144:0/2913352302 reap_dead start 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c10f6d700 1 -- 172.21.15.144:0/2913352302 shutdown_connections 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c10f6d700 1 -- 172.21.15.144:0/2913352302 >> 172.21.15.144:0/2913352302 conn(0x7f6c0c0d0000 msgr2=0x7f6c0c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c10f6d700 1 -- 172.21.15.144:0/2913352302 shutdown_connections 2023-11-28T11:51:54.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.375+0000 7f6c10f6d700 1 -- 172.21.15.144:0/2913352302 wait complete. 2023-11-28T11:51:54.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.376+0000 7f6c10f6d700 1 Processor -- start 2023-11-28T11:51:54.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.376+0000 7f6c10f6d700 1 -- start start 2023-11-28T11:51:54.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6c0c0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c0c150fe0 0x7f6c0c150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c151530 0x7f6c0c150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1591d0 con 0x7f6bf4004850 2023-11-28T11:51:54.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c6670 con 0x7f6c0c150fe0 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c159470 con 0x7f6c0c151530 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6c0c0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6c0c0c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35818/0 (socket says 172.21.15.144:35818) 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 -- 172.21.15.144:0/3288043331 learned_addr learned my addr 172.21.15.144:0/3288043331 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0ad9d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c151530 0x7f6c0c150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 -- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c151530 msgr2=0x7f6c0c150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c151530 0x7f6c0c150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 -- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c0c150fe0 msgr2=0x7f6c0c150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c0c150fe0 0x7f6c0c150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 -- 172.21.15.144:0/3288043331 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c15ccd0 con 0x7f6bf4004850 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c0a59c700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6c0c0c4430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e780 tx=0x7f6bfc00eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:54.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc012590 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc0126f0 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c15d900 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.377+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc012a60 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.378+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c06a270 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.378+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c159960 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.379+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6bfc012bc0 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.379+0000 7f6bfaffd700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6bec0682f0 0x7f6bec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.379+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bfc085c10 con 0x7f6bf4004850 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.379+0000 7f6c09d9b700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6bec0682f0 0x7f6bec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.380+0000 7f6c09d9b700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6bec0682f0 0x7f6bec06a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0cd3d0 tx=0x7f6c00001e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:54.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.383+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6bfc0523f0 con 0x7f6bf4004850 2023-11-28T11:51:54.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.614+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c0c0720a0 con 0x7f6bf4004850 2023-11-28T11:51:54.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.614+0000 7f6bfaffd700 1 -- 172.21.15.144:0/3288043331 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f6bfc055a10 con 0x7f6bf4004850 2023-11-28T11:51:54.616 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:54.617 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:51:16.866858+0000","modified":"2023-11-28T11:51:19.878262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6bec0682f0 msgr2=0x7f6bec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6bec0682f0 0x7f6bec06a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0cd3d0 tx=0x7f6c00001e70 comp rx=0 tx=0).stop 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 msgr2=0x7f6c0c0c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6c0c0c4430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e780 tx=0x7f6bfc00eb40 comp rx=0 tx=0).stop 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 shutdown_connections 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bf4004850 0x7f6c0c0c4430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6bec0682f0 0x7f6bec06a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6c0c151530 0x7f6c0c150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 --2- 172.21.15.144:0/3288043331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c0c150fe0 0x7f6c0c150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 >> 172.21.15.144:0/3288043331 conn(0x7f6c0c0d0000 msgr2=0x7f6c0c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:54.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 shutdown_connections 2023-11-28T11:51:54.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.617+0000 7f6c10f6d700 1 -- 172.21.15.144:0/3288043331 wait complete. 2023-11-28T11:51:54.620 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 8 2023-11-28T11:51:54.631 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:54.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.750+0000 7effc9798700 1 Processor -- start 2023-11-28T11:51:54.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc9798700 1 -- start start 2023-11-28T11:51:54.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc9798700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7effc406fb30 0x7effc40cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc9798700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc406bf10 con 0x7effc4158ce0 2023-11-28T11:51:54.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc9798700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc40ce6f0 con 0x7effc40ce2e0 2023-11-28T11:51:54.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc9798700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc41590b0 con 0x7effc406fb30 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc2ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7effc4158ce0 0x7effc415d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52538/0 (socket says 172.21.15.144:52538) 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.751+0000 7effc2ffd700 1 -- 172.21.15.144:0/2535253643 learned_addr learned my addr 172.21.15.144:0/2535253643 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc27fc700 1 --2- 172.21.15.144:0/2535253643 >> v2:172.21.15.177:3300/0 conn(0x7effc406fb30 0x7effc40cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934025914 0 0) 0x7effc406bf10 con 0x7effc4158ce0 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effac0035b0 con 0x7effc4158ce0 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3506040951 0 0) 0x7effac0035b0 con 0x7effc4158ce0 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 >> v2:172.21.15.177:3300/0 conn(0x7effc406fb30 msgr2=0x7effc40cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 --2- 172.21.15.144:0/2535253643 >> v2:172.21.15.177:3300/0 conn(0x7effc406fb30 0x7effc40cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 >> v1:172.21.15.177:6789/0 conn(0x7effc40ce2e0 legacy=0x7effc406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40cb0d0 con 0x7effc4158ce0 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7effb8001670 con 0x7effc4158ce0 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc27fc700 1 --2- 172.21.15.144:0/2535253643 >> v2:172.21.15.177:3300/0 conn(0x7effc406fb30 0x7effc40cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 >> v1:172.21.15.144:6789/0 conn(0x7effc4158ce0 legacy=0x7effc415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effac003a00 0x7effac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effac0043f0 0x7effac0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effac00cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc41590b0 con 0x7effac0050b0 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac00d110 con 0x7effac003a00 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac00d2d0 con 0x7effac0043f0 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc37fe700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effac00cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc2ffd700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effac003a00 0x7effac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc27fc700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effac0043f0 0x7effac0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc37fe700 1 -- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effac0043f0 msgr2=0x7effac0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:54.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc37fe700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effac0043f0 0x7effac0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc37fe700 1 -- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effac003a00 msgr2=0x7effac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc37fe700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effac003a00 0x7effac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.752+0000 7effc37fe700 1 -- 172.21.15.144:0/2535253643 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac0035b0 con 0x7effac0050b0 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc37fe700 1 -- 172.21.15.144:0/2535253643 reap_dead start 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc37fe700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effac00cb20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c290 tx=0x7effac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed25ef06f65da78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc001e70 con 0x7effac0050b0 2023-11-28T11:51:54.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc00ce70 con 0x7effac0050b0 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc1ffb700 1 -- 172.21.15.144:0/2535253643 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc003c20 con 0x7effac0050b0 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 -- 172.21.15.144:0/2535253643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 msgr2=0x7effac00cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effac00cb20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effbc00c290 tx=0x7effac005880 comp rx=0 tx=0).stop 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 -- 172.21.15.144:0/2535253643 shutdown_connections 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 --2- 172.21.15.144:0/2535253643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effac00cb20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 -- 172.21.15.144:0/2535253643 >> 172.21.15.144:0/2535253643 conn(0x7effc40cb670 msgr2=0x7effc40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:54.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 -- 172.21.15.144:0/2535253643 shutdown_connections 2023-11-28T11:51:54.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.753+0000 7effc9798700 1 -- 172.21.15.144:0/2535253643 wait complete. 2023-11-28T11:51:54.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.755+0000 7effc9798700 1 Processor -- start 2023-11-28T11:51:54.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.755+0000 7effc9798700 1 -- start start 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc9798700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effc40b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc9798700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effc41451a0 0x7effc4145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc9798700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effc40c6c80 0x7effc40c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc9798700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4145b50 con 0x7effac0050b0 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc9798700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc40c54f0 con 0x7effc41451a0 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc9798700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc40c79b0 con 0x7effc40c6c80 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effc40b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effc40b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35838/0 (socket says 172.21.15.144:35838) 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 -- 172.21.15.144:0/497174276 learned_addr learned my addr 172.21.15.144:0/497174276 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc37fe700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effc40c6c80 0x7effc40c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc27fc700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effc41451a0 0x7effc4145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 -- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effc40c6c80 msgr2=0x7effc40c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effc40c6c80 0x7effc40c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 -- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effc41451a0 msgr2=0x7effc4145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effc41451a0 0x7effc4145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc2ffd700 1 -- 172.21.15.144:0/497174276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc406b8c0 con 0x7effac0050b0 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.756+0000 7effc37fe700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effc40c6c80 0x7effc40c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.757+0000 7effc2ffd700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effc40b4560 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7effb8004ad0 tx=0x7effb8004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:54.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.757+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8011b30 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.757+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8011c90 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.757+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc40cb3b0 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.757+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8014370 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.757+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc406bba0 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.758+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc40745c0 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.758+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7effb8013040 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.759+0000 7effab7fe700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effb00682f0 0x7effb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.759+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effb80879b0 con 0x7effac0050b0 2023-11-28T11:51:54.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.759+0000 7effc27fc700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effb00682f0 0x7effb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.759+0000 7effc27fc700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effb00682f0 0x7effb006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effc406d7a0 tx=0x7effb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.763+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7effb8054040 con 0x7effac0050b0 2023-11-28T11:51:54.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.949+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effc406e550 con 0x7effac0050b0 2023-11-28T11:51:54.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.950+0000 7effab7fe700 1 -- 172.21.15.144:0/497174276 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7effb801c090 con 0x7effac0050b0 2023-11-28T11:51:54.952 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:54.952 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:54.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.952+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effb00682f0 msgr2=0x7effb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.952+0000 7effc9798700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effb00682f0 0x7effb006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effc406d7a0 tx=0x7effb4008040 comp rx=0 tx=0).stop 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.952+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 msgr2=0x7effc40b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.952+0000 7effc9798700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effc40b4560 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7effb8004ad0 tx=0x7effb8004e90 comp rx=0 tx=0).stop 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 shutdown_connections 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effac0050b0 0x7effc40b4560 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7effb00682f0 0x7effb006a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effc40c6c80 0x7effc40c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 --2- 172.21.15.144:0/497174276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effc41451a0 0x7effc4145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 >> 172.21.15.144:0/497174276 conn(0x7effc40cb670 msgr2=0x7effc40cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 shutdown_connections 2023-11-28T11:51:54.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:54.953+0000 7effc9798700 1 -- 172.21.15.144:0/497174276 wait complete. 2023-11-28T11:51:54.967 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:51:14.557875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:51:15.833037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:51:54.967 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:51:55.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 Processor -- start 2023-11-28T11:51:55.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 -- start start 2023-11-28T11:51:55.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9318140950 0x7f93180b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f931806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931813ff00 con 0x7f93180cd8c0 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93181404c0 con 0x7f9318140950 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.089+0000 7f931e848700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9318140dc0 con 0x7f93180b2e70 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f931806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f93177fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9318140950 0x7f93180b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f931806f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47932/0 (socket says 172.21.15.144:47932) 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 -- 172.21.15.144:0/2077864195 learned_addr learned my addr 172.21.15.144:0/2077864195 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 -- 172.21.15.144:0/2077864195 >> v2:172.21.15.177:3300/0 conn(0x7f9318140950 msgr2=0x7f93180b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 --2- 172.21.15.144:0/2077864195 >> v2:172.21.15.177:3300/0 conn(0x7f9318140950 0x7f93180b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 -- 172.21.15.144:0/2077864195 >> v1:172.21.15.177:6789/0 conn(0x7f93180cd8c0 legacy=0x7f93180cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 -- 172.21.15.144:0/2077864195 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931815aa80 con 0x7f93180b2e70 2023-11-28T11:51:55.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f93177fe700 1 --2- 172.21.15.144:0/2077864195 >> v2:172.21.15.177:3300/0 conn(0x7f9318140950 0x7f93180b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:55.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.090+0000 7f931cde5700 1 --2- 172.21.15.144:0/2077864195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f931806f850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9310009e50 tx=0x7f931000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13138996833abd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f9316ffd700 1 -- 172.21.15.144:0/2077864195 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000e070 con 0x7f93180b2e70 2023-11-28T11:51:55.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f9316ffd700 1 -- 172.21.15.144:0/2077864195 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000b910 con 0x7f93180b2e70 2023-11-28T11:51:55.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f9316ffd700 1 -- 172.21.15.144:0/2077864195 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000bcc0 con 0x7f93180b2e70 2023-11-28T11:51:55.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 -- 172.21.15.144:0/2077864195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 msgr2=0x7f931806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 --2- 172.21.15.144:0/2077864195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f931806f850 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9310009e50 tx=0x7f931000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 -- 172.21.15.144:0/2077864195 shutdown_connections 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 --2- 172.21.15.144:0/2077864195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f931806f850 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 --2- 172.21.15.144:0/2077864195 >> v2:172.21.15.177:3300/0 conn(0x7f9318140950 0x7f93180b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 -- 172.21.15.144:0/2077864195 >> 172.21.15.144:0/2077864195 conn(0x7f931806aed0 msgr2=0x7f931806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 -- 172.21.15.144:0/2077864195 shutdown_connections 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.091+0000 7f931e848700 1 -- 172.21.15.144:0/2077864195 wait complete. 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.092+0000 7f931e848700 1 Processor -- start 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.093+0000 7f931e848700 1 -- start start 2023-11-28T11:51:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.093+0000 7f931e848700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f93180b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.093+0000 7f931e848700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93180cd8c0 0x7f931815cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931e848700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9318140950 0x7f931815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931e848700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180b4d50 con 0x7f93180cd8c0 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931e848700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318156bd0 con 0x7f9318140950 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931e848700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931815d7d0 con 0x7f93180b2e70 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9317fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f93180b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93180cd8c0 0x7f931815cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9317fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f93180b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47934/0 (socket says 172.21.15.144:47934) 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9317fff700 1 -- 172.21.15.144:0/1637395593 learned_addr learned my addr 172.21.15.144:0/1637395593 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931cde5700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9318140950 0x7f931815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 -- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 msgr2=0x7f93180b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f93180b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 -- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9318140950 msgr2=0x7f931815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9318140950 0x7f931815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 -- 172.21.15.144:0/1637395593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0094a0 con 0x7f93180cd8c0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9317fff700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f93180b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f93177fe700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93180cd8c0 0x7f931815cc80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f930c00e8d0 tx=0x7f930c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c013680 con 0x7f93180cd8c0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c0137e0 con 0x7f93180cd8c0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9310009b30 con 0x7f93180cd8c0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c013b70 con 0x7f93180cd8c0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.094+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180b35f0 con 0x7f93180cd8c0 2023-11-28T11:51:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.095+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93180b3310 con 0x7f93180cd8c0 2023-11-28T11:51:55.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.096+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f930c019030 con 0x7f93180cd8c0 2023-11-28T11:51:55.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.096+0000 7f9314ff9700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f93040682f0 0x7f930406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.096+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f930c085aa0 con 0x7f93180cd8c0 2023-11-28T11:51:55.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.096+0000 7f9317fff700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f93040682f0 0x7f930406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.097+0000 7f9317fff700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f93040682f0 0x7f930406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9308001d00 tx=0x7f9308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.101+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f930c052300 con 0x7f93180cd8c0 2023-11-28T11:51:55.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.281+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9318140150 con 0x7f93180cd8c0 2023-11-28T11:51:55.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.282+0000 7f9314ff9700 1 -- 172.21.15.144:0/1637395593 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f930c055920 con 0x7f93180cd8c0 2023-11-28T11:51:55.284 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f93040682f0 msgr2=0x7f930406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f93040682f0 0x7f930406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9308001d00 tx=0x7f9308009040 comp rx=0 tx=0).stop 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93180cd8c0 msgr2=0x7f931815cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93180cd8c0 0x7f931815cc80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f930c00e8d0 tx=0x7f930c00ebe0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 shutdown_connections 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f93040682f0 0x7f930406a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9318140950 0x7f931815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f93180b2e70 0x7f93180b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 --2- 172.21.15.144:0/1637395593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93180cd8c0 0x7f931815cc80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 >> 172.21.15.144:0/1637395593 conn(0x7f931806aed0 msgr2=0x7f931813db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:55.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 shutdown_connections 2023-11-28T11:51:55.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.284+0000 7f931e848700 1 -- 172.21.15.144:0/1637395593 wait complete. 2023-11-28T11:51:55.298 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:51:55.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3f241700 1 Processor -- start 2023-11-28T11:51:55.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3f241700 1 -- start start 2023-11-28T11:51:55.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3f241700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a381658c0 0x7f6a38167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3f241700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3815d060 con 0x7f6a38071040 2023-11-28T11:51:55.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3f241700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3815d340 con 0x7f6a3806dee0 2023-11-28T11:51:55.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3f241700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38161550 con 0x7f6a381658c0 2023-11-28T11:51:55.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.419+0000 7f6a3cfdd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6a38071040 0x7f6a38071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52572/0 (socket says 172.21.15.144:52572) 2023-11-28T11:51:55.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a3cfdd700 1 -- 172.21.15.144:0/2392534931 learned_addr learned my addr 172.21.15.144:0/2392534931 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a3d7de700 1 --2- 172.21.15.144:0/2392534931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a381658c0 0x7f6a38167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647635856 0 0) 0x7f6a3815d060 con 0x7f6a38071040 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a280035b0 con 0x7f6a38071040 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364056790 0 0) 0x7f6a3815d340 con 0x7f6a3806dee0 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3815d060 con 0x7f6a3806dee0 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435173546 0 0) 0x7f6a280035b0 con 0x7f6a38071040 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a3d7de700 1 -- 172.21.15.144:0/2392534931 >> v1:172.21.15.177:6789/0 conn(0x7f6a3806dee0 legacy=0x7f6a3806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a3d7de700 1 -- 172.21.15.144:0/2392534931 >> v1:172.21.15.144:6789/0 conn(0x7f6a38071040 legacy=0x7f6a38071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a3d7de700 1 -- 172.21.15.144:0/2392534931 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3815d620 con 0x7f6a381658c0 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a3d7de700 1 --2- 172.21.15.144:0/2392534931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a381658c0 0x7f6a38167ca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009dc0 tx=0x7f6a3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf51a06392ae5a58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000e070 con 0x7f6a381658c0 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a30003a20 con 0x7f6a381658c0 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.420+0000 7f6a377fe700 1 -- 172.21.15.144:0/2392534931 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c001670 con 0x7f6a381658c0 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 -- 172.21.15.144:0/2392534931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a381658c0 msgr2=0x7f6a38167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 --2- 172.21.15.144:0/2392534931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a381658c0 0x7f6a38167ca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009dc0 tx=0x7f6a3000b330 comp rx=0 tx=0).stop 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 -- 172.21.15.144:0/2392534931 shutdown_connections 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 --2- 172.21.15.144:0/2392534931 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a381658c0 0x7f6a38167ca0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 -- 172.21.15.144:0/2392534931 >> 172.21.15.144:0/2392534931 conn(0x7f6a380d0000 msgr2=0x7f6a380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 -- 172.21.15.144:0/2392534931 shutdown_connections 2023-11-28T11:51:55.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.421+0000 7f6a3f241700 1 -- 172.21.15.144:0/2392534931 wait complete. 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.422+0000 7f6a3f241700 1 Processor -- start 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.422+0000 7f6a3f241700 1 -- start start 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3f241700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3806dee0 0x7f6a3814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3f241700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 0x7f6a380ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3f241700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a381658c0 0x7f6a380acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3f241700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380ad840 con 0x7f6a38071040 2023-11-28T11:51:55.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3f241700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380adb20 con 0x7f6a381658c0 2023-11-28T11:51:55.429 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3f241700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380ade00 con 0x7f6a3806dee0 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 0x7f6a380ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 0x7f6a380ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35868/0 (socket says 172.21.15.144:35868) 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a37fff700 1 -- 172.21.15.144:0/143981817 learned_addr learned my addr 172.21.15.144:0/143981817 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3cfdd700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3806dee0 0x7f6a3814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.423+0000 7f6a3d7de700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a381658c0 0x7f6a380acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a37fff700 1 -- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3806dee0 msgr2=0x7f6a3814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a37fff700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3806dee0 0x7f6a3814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a37fff700 1 -- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a381658c0 msgr2=0x7f6a380acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a37fff700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a381658c0 0x7f6a380acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a37fff700 1 -- 172.21.15.144:0/143981817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009a70 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a37fff700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 0x7f6a380ae8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c500 tx=0x7f6a2400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240052d0 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a24005450 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3815ccd0 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240038c0 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.424+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3815d900 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.425+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3806a270 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.425+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a24019030 con 0x7f6a38071040 2023-11-28T11:51:55.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.426+0000 7f6a357fa700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a1c0682f0 0x7f6a1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.426+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a24084ec0 con 0x7f6a38071040 2023-11-28T11:51:55.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.426+0000 7f6a3cfdd700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a1c0682f0 0x7f6a1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.426+0000 7f6a3cfdd700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a1c0682f0 0x7f6a1c06a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004a60 tx=0x7f6a2c000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.430+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a24051720 con 0x7f6a38071040 2023-11-28T11:51:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.610+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a380cfae0 con 0x7f6a38071040 2023-11-28T11:51:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.610+0000 7f6a357fa700 1 -- 172.21.15.144:0/143981817 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a240170a0 con 0x7f6a38071040 2023-11-28T11:51:55.613 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a1c0682f0 msgr2=0x7f6a1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a1c0682f0 0x7f6a1c06a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004a60 tx=0x7f6a2c000bc0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 msgr2=0x7f6a380ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 0x7f6a380ae8e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c500 tx=0x7f6a2400c810 comp rx=0 tx=0).stop 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 shutdown_connections 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6a1c0682f0 0x7f6a1c06a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a381658c0 0x7f6a380acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6a3806dee0 0x7f6a3814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 --2- 172.21.15.144:0/143981817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a38071040 0x7f6a380ae8e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 >> 172.21.15.144:0/143981817 conn(0x7f6a380d0000 msgr2=0x7f6a3815f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 shutdown_connections 2023-11-28T11:51:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.613+0000 7f6a3f241700 1 -- 172.21.15.144:0/143981817 wait complete. 2023-11-28T11:51:55.627 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:51:55.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.748+0000 7f92fe16d700 1 Processor -- start 2023-11-28T11:51:55.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92fe16d700 1 -- start start 2023-11-28T11:51:55.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92fe16d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f92f81658c0 0x7f92f8167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92fe16d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f815d060 con 0x7f92f806dee0 2023-11-28T11:51:55.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92fe16d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f815d340 con 0x7f92f8071040 2023-11-28T11:51:55.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92fe16d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f8161550 con 0x7f92f81658c0 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92f6ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f92f806dee0 0x7f92f806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52588/0 (socket says 172.21.15.144:52588) 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.749+0000 7f92f6ffd700 1 -- 172.21.15.144:0/611419551 learned_addr learned my addr 172.21.15.144:0/611419551 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f7fff700 1 --2- 172.21.15.144:0/611419551 >> v2:172.21.15.177:3300/0 conn(0x7f92f81658c0 0x7f92f8167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000195371 0 0) 0x7f92f815d060 con 0x7f92f806dee0 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e40035b0 con 0x7f92f806dee0 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949479751 0 0) 0x7f92e40035b0 con 0x7f92f806dee0 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 >> v2:172.21.15.177:3300/0 conn(0x7f92f81658c0 msgr2=0x7f92f8167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 --2- 172.21.15.144:0/611419551 >> v2:172.21.15.177:3300/0 conn(0x7f92f81658c0 0x7f92f8167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 >> v1:172.21.15.177:6789/0 conn(0x7f92f8071040 legacy=0x7f92f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f815d620 con 0x7f92f806dee0 2023-11-28T11:51:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f92e8001670 con 0x7f92f806dee0 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f7fff700 1 --2- 172.21.15.144:0/611419551 >> v2:172.21.15.177:3300/0 conn(0x7f92f81658c0 0x7f92f8167ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 >> v1:172.21.15.144:6789/0 conn(0x7f92f806dee0 legacy=0x7f92f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e4003a00 0x7f92e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e4004850 0x7f92e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4005320 0x7f92e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8161550 con 0x7f92e4003a00 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e400c250 con 0x7f92e4004850 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e400c410 con 0x7f92e4005320 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e4003a00 0x7f92e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f6ffd700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e4004850 0x7f92e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f7fff700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4005320 0x7f92e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 -- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4005320 msgr2=0x7f92e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4005320 0x7f92e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 -- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e4004850 msgr2=0x7f92e4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e4004850 0x7f92e4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 -- 172.21.15.144:0/611419551 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40035b0 con 0x7f92e4003a00 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f7fff700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4005320 0x7f92e400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f6ffd700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e4004850 0x7f92e4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f7fff700 1 -- 172.21.15.144:0/611419551 reap_dead start 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.750+0000 7f92f77fe700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e4003a00 0x7f92e4003e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f92e0001800 tx=0x7f92e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d8491353ed955d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e00012f0 con 0x7f92e4003a00 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e0005310 con 0x7f92e4003a00 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92f67fc700 1 -- 172.21.15.144:0/611419551 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0006860 con 0x7f92e4003a00 2023-11-28T11:51:55.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 -- 172.21.15.144:0/611419551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e4003a00 msgr2=0x7f92e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e4003a00 0x7f92e4003e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f92e0001800 tx=0x7f92e4005880 comp rx=0 tx=0).stop 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 -- 172.21.15.144:0/611419551 shutdown_connections 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 --2- 172.21.15.144:0/611419551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e4003a00 0x7f92e4003e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 -- 172.21.15.144:0/611419551 >> 172.21.15.144:0/611419551 conn(0x7f92f80d0000 msgr2=0x7f92f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 -- 172.21.15.144:0/611419551 shutdown_connections 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.751+0000 7f92fe16d700 1 -- 172.21.15.144:0/611419551 wait complete. 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.752+0000 7f92fe16d700 1 Processor -- start 2023-11-28T11:51:55.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 -- start start 2023-11-28T11:51:55.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 0x7f92f8151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f81594e0 0x7f92f8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f815e0f0 0x7f92f815e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f8155b60 con 0x7f92f815e0f0 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f815a0b0 con 0x7f92f81594e0 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.753+0000 7f92fe16d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f815eaa0 con 0x7f92e4003a00 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 0x7f92f8151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f81594e0 0x7f92f8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 0x7f92f8151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:47968/0 (socket says 172.21.15.144:47968) 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f77fe700 1 -- 172.21.15.144:0/587646731 learned_addr learned my addr 172.21.15.144:0/587646731 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f815e0f0 0x7f92f815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 -- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 msgr2=0x7f92f8151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 0x7f92f8151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 -- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f81594e0 msgr2=0x7f92f8159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f81594e0 0x7f92f8159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 -- 172.21.15.144:0/587646731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f815ccd0 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f77fe700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 0x7f92f8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92f7fff700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f815e0f0 0x7f92f815e560 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f92f8070fd0 tx=0x7f92f000f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f000fec0 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f0016420 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f815d900 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0016790 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.754+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f806a270 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.755+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f8155ca0 con 0x7f92f815e0f0 2023-11-28T11:51:55.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.756+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92f00168f0 con 0x7f92f815e0f0 2023-11-28T11:51:55.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.756+0000 7f92effff700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92d80682f0 0x7f92d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:55.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.756+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f92f00868f0 con 0x7f92f815e0f0 2023-11-28T11:51:55.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.756+0000 7f92f77fe700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92d80682f0 0x7f92d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:55.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.757+0000 7f92f77fe700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92d80682f0 0x7f92d806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e0014040 tx=0x7f92e0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:55.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.760+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92f8155ca0 con 0x7f92f815e0f0 2023-11-28T11:51:55.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.944+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92f8155ca0 con 0x7f92f815e0f0 2023-11-28T11:51:55.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.944+0000 7f92effff700 1 -- 172.21.15.144:0/587646731 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92f8155ca0 con 0x7f92f815e0f0 2023-11-28T11:51:55.946 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:51:55.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92d80682f0 msgr2=0x7f92d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92d80682f0 0x7f92d806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e0014040 tx=0x7f92e0009030 comp rx=0 tx=0).stop 2023-11-28T11:51:55.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f815e0f0 msgr2=0x7f92f815e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f815e0f0 0x7f92f815e560 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f92f8070fd0 tx=0x7f92f000f660 comp rx=0 tx=0).stop 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 shutdown_connections 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92f815e0f0 0x7f92f815e560 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f92d80682f0 0x7f92d806a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e4003a00 0x7f92f8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 --2- 172.21.15.144:0/587646731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f81594e0 0x7f92f8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 >> 172.21.15.144:0/587646731 conn(0x7f92f80d0000 msgr2=0x7f92f806e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 shutdown_connections 2023-11-28T11:51:55.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:55.946+0000 7f92fe16d700 1 -- 172.21.15.144:0/587646731 wait complete. 2023-11-28T11:51:55.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:51:56.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.081+0000 7f308bc0c700 1 Processor -- start 2023-11-28T11:51:56.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.081+0000 7f308bc0c700 1 -- start start 2023-11-28T11:51:56.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f308bc0c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308406dee0 0x7f308406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f308bc0c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308415d060 con 0x7f3084160540 2023-11-28T11:51:56.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f308bc0c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308415d340 con 0x7f3084071040 2023-11-28T11:51:56.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f308bc0c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308415d620 con 0x7f308406dee0 2023-11-28T11:51:56.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308406dee0 0x7f308406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308406dee0 0x7f308406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35896/0 (socket says 172.21.15.144:35896) 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 -- 172.21.15.144:0/3400946017 learned_addr learned my addr 172.21.15.144:0/3400946017 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 -- 172.21.15.144:0/3400946017 >> v1:172.21.15.177:6789/0 conn(0x7f3084071040 legacy=0x7f3084071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 -- 172.21.15.144:0/3400946017 >> v1:172.21.15.144:6789/0 conn(0x7f3084160540 legacy=0x7f3084160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 -- 172.21.15.144:0/3400946017 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308415d900 con 0x7f308406dee0 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30891a7700 1 --2- 172.21.15.144:0/3400946017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406dee0 0x7f308406e330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f30740098f0 tx=0x7f3074009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4e057f2f604348 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30889a6700 1 -- 172.21.15.144:0/3400946017 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400b8a0 con 0x7f308406dee0 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30889a6700 1 -- 172.21.15.144:0/3400946017 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307400ba00 con 0x7f308406dee0 2023-11-28T11:51:56.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.082+0000 7f30889a6700 1 -- 172.21.15.144:0/3400946017 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307400bd90 con 0x7f308406dee0 2023-11-28T11:51:56.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 -- 172.21.15.144:0/3400946017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406dee0 msgr2=0x7f308406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 --2- 172.21.15.144:0/3400946017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406dee0 0x7f308406e330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f30740098f0 tx=0x7f3074009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:56.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 -- 172.21.15.144:0/3400946017 shutdown_connections 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 --2- 172.21.15.144:0/3400946017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406dee0 0x7f308406e330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 -- 172.21.15.144:0/3400946017 >> 172.21.15.144:0/3400946017 conn(0x7f30840d0000 msgr2=0x7f30840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 -- 172.21.15.144:0/3400946017 shutdown_connections 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.083+0000 7f308bc0c700 1 -- 172.21.15.144:0/3400946017 wait complete. 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.084+0000 7f308bc0c700 1 Processor -- start 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.084+0000 7f308bc0c700 1 -- start start 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308bc0c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f308406dee0 0x7f3084158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308bc0c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3084071040 0x7f30841554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308bc0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 0x7f3084155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308bc0c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840b3d40 con 0x7f3084160540 2023-11-28T11:51:56.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308bc0c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084157040 con 0x7f3084071040 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308bc0c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30840b3f70 con 0x7f308406dee0 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 0x7f3084155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 0x7f3084155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35904/0 (socket says 172.21.15.144:35904) 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 -- 172.21.15.144:0/879540632 learned_addr learned my addr 172.21.15.144:0/879540632 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f30899a8700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f308406dee0 0x7f3084158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f30891a7700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3084071040 0x7f30841554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 -- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f308406dee0 msgr2=0x7f3084158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f308406dee0 0x7f3084158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 -- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3084071040 msgr2=0x7f30841554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3084071040 0x7f30841554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.085+0000 7f308a1a9700 1 -- 172.21.15.144:0/879540632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009550 con 0x7f3084160540 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.086+0000 7f308a1a9700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 0x7f3084155ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cb50 tx=0x7f307c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.086+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c003c00 con 0x7f3084160540 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.086+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c003d60 con 0x7f3084160540 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.086+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308415ccd0 con 0x7f3084160540 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.086+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00aa20 con 0x7f3084160540 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.086+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308406e870 con 0x7f3084160540 2023-11-28T11:51:56.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.087+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308406a270 con 0x7f3084160540 2023-11-28T11:51:56.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.087+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f307c010050 con 0x7f3084160540 2023-11-28T11:51:56.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.087+0000 7f307a7fc700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30680682f0 0x7f306806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.088+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f307c085b20 con 0x7f3084160540 2023-11-28T11:51:56.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.088+0000 7f30899a8700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30680682f0 0x7f306806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.088+0000 7f30899a8700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30680682f0 0x7f306806a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3080000f60 tx=0x7f3080009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.092+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f307c052380 con 0x7f3084160540 2023-11-28T11:51:56.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.275+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30840cd6e0 con 0x7f3084160540 2023-11-28T11:51:56.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.276+0000 7f307a7fc700 1 -- 172.21.15.144:0/879540632 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f307c0559a0 con 0x7f3084160540 2023-11-28T11:51:56.278 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:51:56.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30680682f0 msgr2=0x7f306806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30680682f0 0x7f306806a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3080000f60 tx=0x7f3080009040 comp rx=0 tx=0).stop 2023-11-28T11:51:56.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 msgr2=0x7f3084155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 0x7f3084155ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f307c00cb50 tx=0x7f307c00ce60 comp rx=0 tx=0).stop 2023-11-28T11:51:56.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 shutdown_connections 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3084160540 0x7f3084155ea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f30680682f0 0x7f306806a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f308406dee0 0x7f3084158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 --2- 172.21.15.144:0/879540632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3084071040 0x7f30841554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 >> 172.21.15.144:0/879540632 conn(0x7f30840d0000 msgr2=0x7f30840cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 shutdown_connections 2023-11-28T11:51:56.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.278+0000 7f308bc0c700 1 -- 172.21.15.144:0/879540632 wait complete. 2023-11-28T11:51:56.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:56.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.413+0000 7fd384dfc700 1 Processor -- start 2023-11-28T11:51:56.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.414+0000 7fd384dfc700 1 -- start start 2023-11-28T11:51:56.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.414+0000 7fd384dfc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.414+0000 7fd384dfc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd380160b10 0x7fd3800cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.414+0000 7fd384dfc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd38006b060 0x7fd38006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.414+0000 7fd384dfc700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3800ca720 con 0x7fd38006b060 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd384dfc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3800cc030 con 0x7fd380160b10 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd384dfc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800c9e10 con 0x7fd380160740 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd38006b060 0x7fd38006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd38006b060 0x7fd38006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35912/0 (socket says 172.21.15.144:35912) 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 -- 172.21.15.144:0/4157905520 learned_addr learned my addr 172.21.15.144:0/4157905520 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37dd9b700 1 --2- 172.21.15.144:0/4157905520 >> v2:172.21.15.177:3300/0 conn(0x7fd380160b10 0x7fd3800cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37e59c700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 -- 172.21.15.144:0/4157905520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 msgr2=0x7fd3800cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 -- 172.21.15.144:0/4157905520 >> v2:172.21.15.177:3300/0 conn(0x7fd380160b10 msgr2=0x7fd3800cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 --2- 172.21.15.144:0/4157905520 >> v2:172.21.15.177:3300/0 conn(0x7fd380160b10 0x7fd3800cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 -- 172.21.15.144:0/4157905520 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38006efc0 con 0x7fd38006b060 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37e59c700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37ed9d700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006b4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd38006c680 tx=0x7fd37800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b09bf17ad9b96dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37d59a700 1 -- 172.21.15.144:0/4157905520 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800e070 con 0x7fd38006b060 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37d59a700 1 -- 172.21.15.144:0/4157905520 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800b910 con 0x7fd38006b060 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.415+0000 7fd37d59a700 1 -- 172.21.15.144:0/4157905520 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800bcc0 con 0x7fd38006b060 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 -- 172.21.15.144:0/4157905520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 msgr2=0x7fd38006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006b4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd38006c680 tx=0x7fd37800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 -- 172.21.15.144:0/4157905520 shutdown_connections 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006b4e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 --2- 172.21.15.144:0/4157905520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 --2- 172.21.15.144:0/4157905520 >> v2:172.21.15.177:3300/0 conn(0x7fd380160b10 0x7fd3800cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 -- 172.21.15.144:0/4157905520 >> 172.21.15.144:0/4157905520 conn(0x7fd3800b12b0 msgr2=0x7fd3800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:56.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 -- 172.21.15.144:0/4157905520 shutdown_connections 2023-11-28T11:51:56.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.416+0000 7fd384dfc700 1 -- 172.21.15.144:0/4157905520 wait complete. 2023-11-28T11:51:56.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.418+0000 7fd384dfc700 1 Processor -- start 2023-11-28T11:51:56.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.418+0000 7fd384dfc700 1 -- start start 2023-11-28T11:51:56.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd380160b10 0x7fd3800ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38014c560 con 0x7fd38006b060 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd380155f00 con 0x7fd380160b10 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801561e0 con 0x7fd380160740 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35914/0 (socket says 172.21.15.144:35914) 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 -- 172.21.15.144:0/1732739954 learned_addr learned my addr 172.21.15.144:0/1732739954 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37dd9b700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37ed9d700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd380160b10 0x7fd3800ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.424 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 -- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 msgr2=0x7fd3800ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 -- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd380160b10 msgr2=0x7fd3800ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd380160b10 0x7fd3800ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 -- 172.21.15.144:0/1732739954 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378009b30 con 0x7fd38006b060 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37ed9d700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd380160b10 0x7fd3800ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd37e59c700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006e8d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd37400e810 tx=0x7fd37400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374013610 con 0x7fd38006b060 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374013770 con 0x7fd38006b060 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3800ca3b0 con 0x7fd38006b060 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374013b20 con 0x7fd38006b060 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.419+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3800ca0d0 con 0x7fd38006b060 2023-11-28T11:51:56.425 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.420+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3800cd8d0 con 0x7fd38006b060 2023-11-28T11:51:56.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.421+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd374023070 con 0x7fd38006b060 2023-11-28T11:51:56.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.421+0000 7fd36effd700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd3600682f0 0x7fd36006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.421+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd374085920 con 0x7fd38006b060 2023-11-28T11:51:56.426 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.421+0000 7fd37dd9b700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd3600682f0 0x7fd36006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.421+0000 7fd37dd9b700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd3600682f0 0x7fd36006a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3801f6430 tx=0x7fd370008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.425+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd374052180 con 0x7fd38006b060 2023-11-28T11:51:56.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.650+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3801615c0 con 0x7fd38006b060 2023-11-28T11:51:56.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.651+0000 7fd36effd700 1 -- 172.21.15.144:0/1732739954 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fd3740557a0 con 0x7fd38006b060 2023-11-28T11:51:56.653 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:56.653 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:51:16.866858+0000","modified":"2023-11-28T11:51:19.878262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd3600682f0 msgr2=0x7fd36006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd3600682f0 0x7fd36006a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3801f6430 tx=0x7fd370008040 comp rx=0 tx=0).stop 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 msgr2=0x7fd38006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006e8d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd37400e810 tx=0x7fd37400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 shutdown_connections 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd38006b060 0x7fd38006e8d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd3600682f0 0x7fd36006a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd380160b10 0x7fd3800ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 --2- 172.21.15.144:0/1732739954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd380160740 0x7fd3800ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 >> 172.21.15.144:0/1732739954 conn(0x7fd3800b12b0 msgr2=0x7fd3800ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:56.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 shutdown_connections 2023-11-28T11:51:56.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.653+0000 7fd384dfc700 1 -- 172.21.15.144:0/1732739954 wait complete. 2023-11-28T11:51:56.657 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 8 2023-11-28T11:51:56.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:56.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.796+0000 7efee1001700 1 Processor -- start 2023-11-28T11:51:56.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.796+0000 7efee1001700 1 -- start start 2023-11-28T11:51:56.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.796+0000 7efee1001700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efedc154cf0 0x7efedc1404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.796+0000 7efee1001700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc140a30 0x7efedc06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efee1001700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efedc06eb00 0x7efedc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efee1001700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc1550c0 con 0x7efedc06eb00 2023-11-28T11:51:56.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efee1001700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc06ef80 con 0x7efedc154cf0 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efee1001700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc15ee00 con 0x7efedc140a30 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efedc06eb00 0x7efedc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efedc06eb00 0x7efedc070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35926/0 (socket says 172.21.15.144:35926) 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 -- 172.21.15.144:0/3778441885 learned_addr learned my addr 172.21.15.144:0/3778441885 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efeda59c700 1 --2- 172.21.15.144:0/3778441885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc140a30 0x7efedc06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedad9d700 1 --2- 172.21.15.144:0/3778441885 >> v2:172.21.15.177:3300/0 conn(0x7efedc154cf0 0x7efedc1404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 -- 172.21.15.144:0/3778441885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc140a30 msgr2=0x7efedc06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 --2- 172.21.15.144:0/3778441885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc140a30 0x7efedc06e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 -- 172.21.15.144:0/3778441885 >> v2:172.21.15.177:3300/0 conn(0x7efedc154cf0 msgr2=0x7efedc1404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:56.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 --2- 172.21.15.144:0/3778441885 >> v2:172.21.15.177:3300/0 conn(0x7efedc154cf0 0x7efedc1404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 -- 172.21.15.144:0/3778441885 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc15e1f0 con 0x7efedc06eb00 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efedb59e700 1 --2- 172.21.15.144:0/3778441885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc070b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cd580 tx=0x7efed400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8fb14b33455fe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efed9d9b700 1 -- 172.21.15.144:0/3778441885 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400e070 con 0x7efedc06eb00 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efed9d9b700 1 -- 172.21.15.144:0/3778441885 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed400b9c0 con 0x7efedc06eb00 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.797+0000 7efed9d9b700 1 -- 172.21.15.144:0/3778441885 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400bd70 con 0x7efedc06eb00 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 -- 172.21.15.144:0/3778441885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 msgr2=0x7efedc070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 --2- 172.21.15.144:0/3778441885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc070b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cd580 tx=0x7efed400b330 comp rx=0 tx=0).stop 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 -- 172.21.15.144:0/3778441885 shutdown_connections 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 --2- 172.21.15.144:0/3778441885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc070b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 --2- 172.21.15.144:0/3778441885 >> v2:172.21.15.177:3300/0 conn(0x7efedc154cf0 0x7efedc1404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 --2- 172.21.15.144:0/3778441885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc140a30 0x7efedc06e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 -- 172.21.15.144:0/3778441885 >> 172.21.15.144:0/3778441885 conn(0x7efedc0cc6e0 msgr2=0x7efedc0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 -- 172.21.15.144:0/3778441885 shutdown_connections 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.798+0000 7efee1001700 1 -- 172.21.15.144:0/3778441885 wait complete. 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.799+0000 7efee1001700 1 Processor -- start 2023-11-28T11:51:56.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.799+0000 7efee1001700 1 -- start start 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efee1001700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc1f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efee1001700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efedc140a30 0x7efedc1f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efee1001700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc154cf0 0x7efedc15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efee1001700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc1f2ea0 con 0x7efedc06eb00 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efee1001700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0c4090 con 0x7efedc140a30 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efee1001700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc1f1d80 con 0x7efedc154cf0 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efedad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc1f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efedad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc1f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35932/0 (socket says 172.21.15.144:35932) 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efedad9d700 1 -- 172.21.15.144:0/3530071680 learned_addr learned my addr 172.21.15.144:0/3530071680 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efedb59e700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc154cf0 0x7efedc15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.800+0000 7efeda59c700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efedc140a30 0x7efedc1f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efedad9d700 1 -- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc154cf0 msgr2=0x7efedc15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efedad9d700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc154cf0 0x7efedc15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efedad9d700 1 -- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efedc140a30 msgr2=0x7efedc1f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efedad9d700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efedc140a30 0x7efedc1f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efedad9d700 1 -- 172.21.15.144:0/3530071680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009b30 con 0x7efedc06eb00 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efedad9d700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc1f2960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efecc00d860 tx=0x7efecc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0125f0 con 0x7efedc06eb00 2023-11-28T11:51:56.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc012750 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc15e7b0 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc012b00 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.801+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc15e4d0 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.802+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc15ea90 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.802+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efecc012c60 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.803+0000 7efec37fe700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efec80682f0 0x7efec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.803+0000 7efeda59c700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efec80682f0 0x7efec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.803+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efecc085e90 con 0x7efedc06eb00 2023-11-28T11:51:56.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.803+0000 7efeda59c700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efec80682f0 0x7efec806a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cf7f0 tx=0x7efed0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:56.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.807+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efecc052670 con 0x7efedc06eb00 2023-11-28T11:51:56.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.996+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efedc061e80 con 0x7efedc06eb00 2023-11-28T11:51:56.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.997+0000 7efec37fe700 1 -- 172.21.15.144:0/3530071680 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7efecc055c90 con 0x7efedc06eb00 2023-11-28T11:51:56.999 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:57.000 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:57.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.999+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efec80682f0 msgr2=0x7efec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.999+0000 7efee1001700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efec80682f0 0x7efec806a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cf7f0 tx=0x7efed0008040 comp rx=0 tx=0).stop 2023-11-28T11:51:57.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.999+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 msgr2=0x7efedc1f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.999+0000 7efee1001700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc1f2960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efecc00d860 tx=0x7efecc00db70 comp rx=0 tx=0).stop 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:56.999+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 shutdown_connections 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efedc06eb00 0x7efedc1f2960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efec80682f0 0x7efec806a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efedc154cf0 0x7efedc15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 --2- 172.21.15.144:0/3530071680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efedc140a30 0x7efedc1f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 >> 172.21.15.144:0/3530071680 conn(0x7efedc0cc6e0 msgr2=0x7efedc0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 shutdown_connections 2023-11-28T11:51:57.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.000+0000 7efee1001700 1 -- 172.21.15.144:0/3530071680 wait complete. 2023-11-28T11:51:57.015 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:51:57.015 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:57.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.135+0000 7fe585830700 1 Processor -- start 2023-11-28T11:51:57.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.135+0000 7fe585830700 1 -- start start 2023-11-28T11:51:57.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.135+0000 7fe585830700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 0x7fe5800cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.135+0000 7fe585830700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe580160b10 0x7fe5800cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe585830700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58006bbf0 con 0x7fe58006b060 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe585830700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5800ca720 con 0x7fe580160b10 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe585830700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800cc030 con 0x7fe580160740 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57f7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe58006b060 0x7fe58006b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52652/0 (socket says 172.21.15.144:52652) 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57f7fe700 1 -- 172.21.15.144:0/3630510489 learned_addr learned my addr 172.21.15.144:0/3630510489 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57effd700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 0x7fe5800cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57e7fc700 1 --2- 172.21.15.144:0/3630510489 >> v2:172.21.15.177:3300/0 conn(0x7fe580160b10 0x7fe5800cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415357783 0 0) 0x7fe58006bbf0 con 0x7fe58006b060 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5680035b0 con 0x7fe58006b060 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 224989056 0 0) 0x7fe5680035b0 con 0x7fe58006b060 2023-11-28T11:51:57.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 msgr2=0x7fe5800cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 0x7fe5800cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 >> v2:172.21.15.177:3300/0 conn(0x7fe580160b10 msgr2=0x7fe5800cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 --2- 172.21.15.144:0/3630510489 >> v2:172.21.15.177:3300/0 conn(0x7fe580160b10 0x7fe5800cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800c9e10 con 0x7fe58006b060 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fe5780019f0 con 0x7fe58006b060 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 >> v1:172.21.15.144:6789/0 conn(0x7fe58006b060 legacy=0x7fe58006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe568003a00 0x7fe568003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 0x7fe568004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d90 0x7fe568005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800cc030 con 0x7fe568004d90 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56800ba10 con 0x7fe568003a00 2023-11-28T11:51:57.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56800bb50 con 0x7fe580160740 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d90 0x7fe568005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57e7fc700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 0x7fe568004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.136+0000 7fe57effd700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe568003a00 0x7fe568003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57f7fe700 1 -- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 msgr2=0x7fe568004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe580160740 0x7fe568004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57f7fe700 1 -- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe568003a00 msgr2=0x7fe568003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe568003a00 0x7fe568003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57f7fe700 1 -- 172.21.15.144:0/3630510489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680035b0 con 0x7fe568004d90 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d90 0x7fe568005200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe57800a8d0 tx=0x7fe568005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3221253973cbb29 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800a340 con 0x7fe568004d90 2023-11-28T11:51:57.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578012600 con 0x7fe568004d90 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.137+0000 7fe57dffb700 1 -- 172.21.15.144:0/3630510489 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5780129b0 con 0x7fe568004d90 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe585830700 1 -- 172.21.15.144:0/3630510489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d90 msgr2=0x7fe568005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe585830700 1 --2- 172.21.15.144:0/3630510489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d90 0x7fe568005200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe57800a8d0 tx=0x7fe568005880 comp rx=0 tx=0).stop 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe57f7fe700 1 -- 172.21.15.144:0/3630510489 reap_dead start 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe585830700 1 -- 172.21.15.144:0/3630510489 shutdown_connections 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe585830700 1 -- 172.21.15.144:0/3630510489 >> 172.21.15.144:0/3630510489 conn(0x7fe5800b12b0 msgr2=0x7fe5800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe585830700 1 -- 172.21.15.144:0/3630510489 shutdown_connections 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.138+0000 7fe585830700 1 -- 172.21.15.144:0/3630510489 wait complete. 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.139+0000 7fe585830700 1 Processor -- start 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.139+0000 7fe585830700 1 -- start start 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe585830700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe568004d90 0x7fe580151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe585830700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe580144dd0 0x7fe580155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe585830700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 0x7fe58015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe585830700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58015fd30 con 0x7fe58015f380 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe585830700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe580158520 con 0x7fe580144dd0 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe585830700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5801562d0 con 0x7fe568004d90 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 0x7fe58015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 0x7fe58015f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35952/0 (socket says 172.21.15.144:35952) 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 -- 172.21.15.144:0/1516185249 learned_addr learned my addr 172.21.15.144:0/1516185249 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57e7fc700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe580144dd0 0x7fe580155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57effd700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe568004d90 0x7fe580151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 -- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe568004d90 msgr2=0x7fe580151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe568004d90 0x7fe580151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 -- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe580144dd0 msgr2=0x7fe580155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe580144dd0 0x7fe580155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 -- 172.21.15.144:0/1516185249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800ca0d0 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57e7fc700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe580144dd0 0x7fe580155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe57f7fe700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 0x7fe58015f7f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe57800aec0 tx=0x7fe578001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.140+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800a340 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.141+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5780019c0 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.141+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58006efc0 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.141+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578026600 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.141+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5800ca3b0 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.141+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5800adad0 con 0x7fe58015f380 2023-11-28T11:51:57.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.142+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe578010070 con 0x7fe58015f380 2023-11-28T11:51:57.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.143+0000 7fe567fff700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe56c0682f0 0x7fe56c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.143+0000 7fe57effd700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe56c0682f0 0x7fe56c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.143+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe578087b40 con 0x7fe58015f380 2023-11-28T11:51:57.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.143+0000 7fe57effd700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe56c0682f0 0x7fe56c06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe58015a1f0 tx=0x7fe570008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.147+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5780543c0 con 0x7fe58015f380 2023-11-28T11:51:57.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.369+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5800745c0 con 0x7fe58015f380 2023-11-28T11:51:57.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.370+0000 7fe567fff700 1 -- 172.21.15.144:0/1516185249 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7fe578017020 con 0x7fe58015f380 2023-11-28T11:51:57.375 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:57.375 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:51:16.866858+0000","modified":"2023-11-28T11:51:19.878262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:51:57.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe56c0682f0 msgr2=0x7fe56c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe56c0682f0 0x7fe56c06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe58015a1f0 tx=0x7fe570008040 comp rx=0 tx=0).stop 2023-11-28T11:51:57.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 msgr2=0x7fe58015f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 0x7fe58015f7f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe57800aec0 tx=0x7fe578001360 comp rx=0 tx=0).stop 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 shutdown_connections 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58015f380 0x7fe58015f7f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe56c0682f0 0x7fe56c06a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.371+0000 7fe585830700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe568004d90 0x7fe580151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.372+0000 7fe585830700 1 --2- 172.21.15.144:0/1516185249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe580144dd0 0x7fe580155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.372+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 >> 172.21.15.144:0/1516185249 conn(0x7fe5800b12b0 msgr2=0x7fe5800cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:57.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.372+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 shutdown_connections 2023-11-28T11:51:57.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.372+0000 7fe585830700 1 -- 172.21.15.144:0/1516185249 wait complete. 2023-11-28T11:51:57.378 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 8 2023-11-28T11:51:57.386 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:51:57.507 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.504+0000 7f1073eba700 1 Processor -- start 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.504+0000 7f1073eba700 1 -- start start 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1073eba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 0x7f106c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1073eba700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f106c06dee0 0x7f106c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1073eba700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c071950 con 0x7f106c06cdc0 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1073eba700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c161550 con 0x7f106c06dee0 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1073eba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15d060 con 0x7f106c071040 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 0x7f106c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071455700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f106c06dee0 0x7f106c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 0x7f106c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:48038/0 (socket says 172.21.15.144:48038) 2023-11-28T11:51:57.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071455700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f106c06dee0 0x7f106c06e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:38230/0 (socket says 172.21.15.144:38230) 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1072457700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f106c06cdc0 0x7f106c06d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:53160/0 (socket says 172.21.15.144:53160) 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 -- 172.21.15.144:0/261798537 learned_addr learned my addr 172.21.15.144:0/261798537 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 -- 172.21.15.144:0/261798537 >> v2:172.21.15.177:3300/0 conn(0x7f106c06dee0 msgr2=0x7f106c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 --2- 172.21.15.144:0/261798537 >> v2:172.21.15.177:3300/0 conn(0x7f106c06dee0 0x7f106c06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 -- 172.21.15.144:0/261798537 >> v1:172.21.15.177:6789/0 conn(0x7f106c06cdc0 legacy=0x7f106c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.505+0000 7f1071c56700 1 -- 172.21.15.144:0/261798537 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c15d340 con 0x7f106c071040 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1071c56700 1 --2- 172.21.15.144:0/261798537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 0x7f106c071410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10680098f0 tx=0x7f1068009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff57221a65a74336 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1070c54700 1 -- 172.21.15.144:0/261798537 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068011830 con 0x7f106c071040 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1070c54700 1 -- 172.21.15.144:0/261798537 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068011990 con 0x7f106c071040 2023-11-28T11:51:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1070c54700 1 -- 172.21.15.144:0/261798537 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068011d20 con 0x7f106c071040 2023-11-28T11:51:57.512 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 -- 172.21.15.144:0/261798537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 msgr2=0x7f106c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 --2- 172.21.15.144:0/261798537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 0x7f106c071410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10680098f0 tx=0x7f1068009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 -- 172.21.15.144:0/261798537 shutdown_connections 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 --2- 172.21.15.144:0/261798537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c071040 0x7f106c071410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 --2- 172.21.15.144:0/261798537 >> v2:172.21.15.177:3300/0 conn(0x7f106c06dee0 0x7f106c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 -- 172.21.15.144:0/261798537 >> 172.21.15.144:0/261798537 conn(0x7f106c0d0000 msgr2=0x7f106c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 -- 172.21.15.144:0/261798537 shutdown_connections 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.506+0000 7f1073eba700 1 -- 172.21.15.144:0/261798537 wait complete. 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 Processor -- start 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 -- start start 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c06cdc0 0x7f106c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 0x7f106c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f106c071040 0x7f106c153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0c6a90 con 0x7f106c06dee0 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c153e70 con 0x7f106c071040 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1073eba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c0cf750 con 0x7f106c06cdc0 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1071455700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 0x7f106c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1071455700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 0x7f106c1533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35960/0 (socket says 172.21.15.144:35960) 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1071455700 1 -- 172.21.15.144:0/738777560 learned_addr learned my addr 172.21.15.144:0/738777560 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.508+0000 7f1072457700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f106c071040 0x7f106c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071c56700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c06cdc0 0x7f106c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071455700 1 -- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c06cdc0 msgr2=0x7f106c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071455700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c06cdc0 0x7f106c157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071455700 1 -- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f106c071040 msgr2=0x7f106c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071455700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f106c071040 0x7f106c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071455700 1 -- 172.21.15.144:0/738777560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009550 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1071455700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 0x7f106c1533f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e870 tx=0x7f105c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c009590 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c013680 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c15d900 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0139f0 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.509+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c15d620 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.510+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106c06e890 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.510+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f105c019030 con 0x7f106c06dee0 2023-11-28T11:51:57.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.511+0000 7f10627fc700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10500682f0 0x7f105006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.511+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f105c085950 con 0x7f106c06dee0 2023-11-28T11:51:57.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.511+0000 7f1071c56700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10500682f0 0x7f105006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.511+0000 7f1071c56700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10500682f0 0x7f105006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1068009840 tx=0x7f1068008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.515+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f105c0521b0 con 0x7f106c06dee0 2023-11-28T11:51:57.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.738+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f106c0c6810 con 0x7f106c06dee0 2023-11-28T11:51:57.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.738+0000 7f10627fc700 1 -- 172.21.15.144:0/738777560 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4648 (secure 0 0 0) 0x7f105c0557d0 con 0x7f106c06dee0 2023-11-28T11:51:57.740 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:57.740 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/786983041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":786983041},{"type":"v1","addr":"172.21.15.177:6835","nonce":786983041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3821400257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3821400257},{"type":"v1","addr":"172.21.15.144:6836","nonce":3821400257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2464739406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2464739406},{"type":"v1","addr":"172.21.15.144:6837","nonce":2464739406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:51:16.866858+0000","modified":"2023-11-28T11:51:19.878262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6833/144294733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":144294733},{"type":"v1","addr":"172.21.15.177:6833","nonce":144294733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10500682f0 msgr2=0x7f105006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10500682f0 0x7f105006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1068009840 tx=0x7f1068008030 comp rx=0 tx=0).stop 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 msgr2=0x7f106c1533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 0x7f106c1533f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e870 tx=0x7f105c00eb80 comp rx=0 tx=0).stop 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 shutdown_connections 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10500682f0 0x7f105006a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f106c071040 0x7f106c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f106c06cdc0 0x7f106c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 --2- 172.21.15.144:0/738777560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f106c06dee0 0x7f106c1533f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 >> 172.21.15.144:0/738777560 conn(0x7f106c0d0000 msgr2=0x7f106c0cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:57.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 shutdown_connections 2023-11-28T11:51:57.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.740+0000 7f1073eba700 1 -- 172.21.15.144:0/738777560 wait complete. 2023-11-28T11:51:57.745 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 8 2023-11-28T11:51:57.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:51:57.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.873+0000 7f0105073700 1 Processor -- start 2023-11-28T11:51:57.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.873+0000 7f0105073700 1 -- start start 2023-11-28T11:51:57.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f0105073700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f01000cbdf0 0x7f01000af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f0105073700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000cc5f0 con 0x7f010015cef0 2023-11-28T11:51:57.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f0105073700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015d470 con 0x7f010006ba90 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f0105073700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010015d750 con 0x7f01000cbdf0 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f00ff59e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f01000cbdf0 0x7f01000af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f00ff59e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f01000cbdf0 0x7f01000af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35972/0 (socket says 172.21.15.144:35972) 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.874+0000 7f00ff59e700 1 -- 172.21.15.144:0/3832409762 learned_addr learned my addr 172.21.15.144:0/3832409762 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00ff59e700 1 -- 172.21.15.144:0/3832409762 >> v1:172.21.15.177:6789/0 conn(0x7f010006ba90 legacy=0x7f01000ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00ff59e700 1 -- 172.21.15.144:0/3832409762 >> v1:172.21.15.144:6789/0 conn(0x7f010015cef0 legacy=0x7f010006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00ff59e700 1 -- 172.21.15.144:0/3832409762 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010015da30 con 0x7f01000cbdf0 2023-11-28T11:51:57.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00ff59e700 1 --2- 172.21.15.144:0/3832409762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000af400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009e50 tx=0x7f00f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15f7151dc971ef5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00fdd9b700 1 -- 172.21.15.144:0/3832409762 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800e070 con 0x7f01000cbdf0 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00fdd9b700 1 -- 172.21.15.144:0/3832409762 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f8003a20 con 0x7f01000cbdf0 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f00fdd9b700 1 -- 172.21.15.144:0/3832409762 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8003dd0 con 0x7f01000cbdf0 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f0105073700 1 -- 172.21.15.144:0/3832409762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 msgr2=0x7f01000af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f0105073700 1 --2- 172.21.15.144:0/3832409762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000af400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009e50 tx=0x7f00f800b330 comp rx=0 tx=0).stop 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f0105073700 1 -- 172.21.15.144:0/3832409762 shutdown_connections 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f0105073700 1 --2- 172.21.15.144:0/3832409762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000af400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.875+0000 7f0105073700 1 -- 172.21.15.144:0/3832409762 >> 172.21.15.144:0/3832409762 conn(0x7f01000d0000 msgr2=0x7f01000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.876+0000 7f0105073700 1 -- 172.21.15.144:0/3832409762 shutdown_connections 2023-11-28T11:51:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.876+0000 7f0105073700 1 -- 172.21.15.144:0/3832409762 wait complete. 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.877+0000 7f0105073700 1 Processor -- start 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.877+0000 7f0105073700 1 -- start start 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f0105073700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f010006ba90 0x7f01001efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f0105073700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f0105073700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f010015cef0 0x7f01000b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f0105073700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c82c0 con 0x7f01000cbdf0 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f0105073700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c85a0 con 0x7f010006ba90 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f0105073700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000c8880 con 0x7f010015cef0 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35976/0 (socket says 172.21.15.144:35976) 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 -- 172.21.15.144:0/1048379977 learned_addr learned my addr 172.21.15.144:0/1048379977 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00ff59e700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f010015cef0 0x7f01000b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fed9d700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f010006ba90 0x7f01001efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 -- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f010015cef0 msgr2=0x7f01000b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f010015cef0 0x7f01000b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 -- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f010006ba90 msgr2=0x7f01001efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f010006ba90 0x7f01001efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 -- 172.21.15.144:0/1048379977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f8009b30 con 0x7f01000cbdf0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00fe59c700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000b4e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f00f000c4d0 tx=0x7f00f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.878+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0011790 con 0x7f01000cbdf0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.879+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f00118f0 con 0x7f01000cbdf0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.879+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010015dff0 con 0x7f01000cbdf0 2023-11-28T11:51:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.879+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0011ca0 con 0x7f01000cbdf0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.879+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010015dd10 con 0x7f01000cbdf0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.879+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01000cad60 con 0x7f01000cbdf0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.880+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f00f0010050 con 0x7f01000cbdf0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.880+0000 7f00e77fe700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f00ec0682f0 0x7f00ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.881+0000 7f00fed9d700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f00ec0682f0 0x7f00ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.881+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00f0085a70 con 0x7f01000cbdf0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.881+0000 7f00fed9d700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f00ec0682f0 0x7f00ec06a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f00f4001580 tx=0x7f00f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:57.885+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00f00521e0 con 0x7f01000cbdf0 2023-11-28T11:51:58.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.070+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01000af310 con 0x7f01000cbdf0 2023-11-28T11:51:58.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.071+0000 7f00e77fe700 1 -- 172.21.15.144:0/1048379977 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17897 (secure 0 0 0) 0x7f00f0055800 con 0x7f01000cbdf0 2023-11-28T11:51:58.073 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:51:58.073 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:51:17.868511+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:51:14.557875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:51:15.833037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f00ec0682f0 msgr2=0x7f00ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f00ec0682f0 0x7f00ec06a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f00f4001580 tx=0x7f00f4009040 comp rx=0 tx=0).stop 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 msgr2=0x7f01000b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000b4e40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f00f000c4d0 tx=0x7f00f000c890 comp rx=0 tx=0).stop 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 shutdown_connections 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f00ec0682f0 0x7f00ec06a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f010015cef0 0x7f01000b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f010006ba90 0x7f01001efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 --2- 172.21.15.144:0/1048379977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01000cbdf0 0x7f01000b4e40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 >> 172.21.15.144:0/1048379977 conn(0x7f01000d0000 msgr2=0x7f010006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:58.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 shutdown_connections 2023-11-28T11:51:58.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.073+0000 7f0105073700 1 -- 172.21.15.144:0/1048379977 wait complete. 2023-11-28T11:51:58.087 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:51:58.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.206+0000 7fb2c1874700 1 Processor -- start 2023-11-28T11:51:58.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.206+0000 7fb2c1874700 1 -- start start 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2c1874700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2bc0b3090 0x7fb2bc0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2c1874700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb2bc0cb620 0x7fb2bc0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2c1874700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0cf4d0 con 0x7fb2bc0cf070 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2c1874700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc15ad60 con 0x7fb2bc0b3090 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2c1874700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0adb90 con 0x7fb2bc0cb620 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2bc0b3090 0x7fb2bc0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2bc0b3090 0x7fb2bc0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35982/0 (socket says 172.21.15.144:35982) 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 -- 172.21.15.144:0/2256160570 learned_addr learned my addr 172.21.15.144:0/2256160570 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2ba7fc700 1 --2- 172.21.15.144:0/2256160570 >> v2:172.21.15.177:3300/0 conn(0x7fb2bc0cb620 0x7fb2bc0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 -- 172.21.15.144:0/2256160570 >> v2:172.21.15.177:3300/0 conn(0x7fb2bc0cb620 msgr2=0x7fb2bc0ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 --2- 172.21.15.144:0/2256160570 >> v2:172.21.15.177:3300/0 conn(0x7fb2bc0cb620 0x7fb2bc0ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 -- 172.21.15.144:0/2256160570 >> v1:172.21.15.177:6789/0 conn(0x7fb2bc0cf070 legacy=0x7fb2bc0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 -- 172.21.15.144:0/2256160570 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0ccdd0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.207+0000 7fb2baffd700 1 --2- 172.21.15.144:0/2256160570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc0cc4a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0672f0 tx=0x7fb2b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4101fe5acfbd1ef0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2b9ffb700 1 -- 172.21.15.144:0/2256160570 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004230 con 0x7fb2bc0b3090 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2b9ffb700 1 -- 172.21.15.144:0/2256160570 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b0004390 con 0x7fb2bc0b3090 2023-11-28T11:51:58.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2b9ffb700 1 -- 172.21.15.144:0/2256160570 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00036a0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 -- 172.21.15.144:0/2256160570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 msgr2=0x7fb2bc0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 --2- 172.21.15.144:0/2256160570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc0cc4a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0672f0 tx=0x7fb2b0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 -- 172.21.15.144:0/2256160570 shutdown_connections 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 --2- 172.21.15.144:0/2256160570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc0cc4a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 --2- 172.21.15.144:0/2256160570 >> v2:172.21.15.177:3300/0 conn(0x7fb2bc0cb620 0x7fb2bc0ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 -- 172.21.15.144:0/2256160570 >> 172.21.15.144:0/2256160570 conn(0x7fb2bc09a230 msgr2=0x7fb2bc08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 -- 172.21.15.144:0/2256160570 shutdown_connections 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.208+0000 7fb2c1874700 1 -- 172.21.15.144:0/2256160570 wait complete. 2023-11-28T11:51:58.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.209+0000 7fb2c1874700 1 Processor -- start 2023-11-28T11:51:58.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 -- start start 2023-11-28T11:51:58.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2bc0cb620 0x7fb2bc06b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2bc0cf070 0x7fb2bc0b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc156c20 con 0x7fb2bc0b3090 2023-11-28T11:51:58.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06c710 con 0x7fb2bc0cf070 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2c1874700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06c150 con 0x7fb2bc0cb620 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2baffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2baffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc1566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35988/0 (socket says 172.21.15.144:35988) 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2baffd700 1 -- 172.21.15.144:0/1948560467 learned_addr learned my addr 172.21.15.144:0/1948560467 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2ba7fc700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2bc0cb620 0x7fb2bc06b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.210+0000 7fb2bb7fe700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2bc0cf070 0x7fb2bc0b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2baffd700 1 -- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2bc0cb620 msgr2=0x7fb2bc06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2baffd700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2bc0cb620 0x7fb2bc06b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2baffd700 1 -- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2bc0cf070 msgr2=0x7fb2bc0b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2baffd700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2bc0cf070 0x7fb2bc0b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2baffd700 1 -- 172.21.15.144:0/1948560467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009550 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2baffd700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc1566e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009c10 tx=0x7fb2b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003c10 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b001f070 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc0cd390 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0011630 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.211+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc0cd0b0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.212+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc0adde0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.212+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb2b0017020 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.213+0000 7fb2a3fff700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2a80682f0 0x7fb2a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.213+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2b00872d0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.213+0000 7fb2ba7fc700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2a80682f0 0x7fb2a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.213+0000 7fb2ba7fc700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2a80682f0 0x7fb2a806a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc071a70 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:58.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.217+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb2b0053ab0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.439+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb2bc0745f0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.778 INFO:tasks.ceph.mon.a.smithi144.stderr:2023-11-28T11:51:58.777+0000 7fdb2a043700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:51:58.786 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.785+0000 7fb2a3fff700 1 -- 172.21.15.144:0/1948560467 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb2b00570d0 con 0x7fb2bc0b3090 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2a80682f0 msgr2=0x7fb2a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2a80682f0 0x7fb2a806a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc071a70 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 msgr2=0x7fb2bc1566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc1566e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009c10 tx=0x7fb2b0004750 comp rx=0 tx=0).stop 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 shutdown_connections 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2bc0b3090 0x7fb2bc1566e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2a80682f0 0x7fb2a806a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2bc0cf070 0x7fb2bc0b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 --2- 172.21.15.144:0/1948560467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2bc0cb620 0x7fb2bc06b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 >> 172.21.15.144:0/1948560467 conn(0x7fb2bc09a230 msgr2=0x7fb2bc1e4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:58.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 shutdown_connections 2023-11-28T11:51:58.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.787+0000 7fb2c1874700 1 -- 172.21.15.144:0/1948560467 wait complete. 2023-11-28T11:51:58.790 INFO:teuthology.orchestra.run.smithi144.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:51:58.802 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:51:58.829 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T11:51:58.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.923+0000 7f6d2ae24700 1 Processor -- start 2023-11-28T11:51:58.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.923+0000 7f6d2ae24700 1 -- start start 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.924+0000 7f6d2ae24700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6d2406ba90 0x7f6d240af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.924+0000 7f6d2ae24700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d240cc5f0 con 0x7f6d2415cef0 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.924+0000 7f6d2ae24700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2415d470 con 0x7f6d240cbdf0 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.924+0000 7f6d2ae24700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2415d750 con 0x7f6d2406ba90 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6d2406ba90 0x7f6d240af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6d2406ba90 0x7f6d240af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35996/0 (socket says 172.21.15.144:35996) 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 -- 172.21.15.144:0/3607440102 learned_addr learned my addr 172.21.15.144:0/3607440102 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:58.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d28bc0700 1 --1- 172.21.15.144:0/3607440102 >> v1:172.21.15.144:6789/0 conn(0x7f6d2415cef0 0x7f6d2406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52708/0 (socket says 172.21.15.144:52708) 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 -- 172.21.15.144:0/3607440102 >> v1:172.21.15.177:6789/0 conn(0x7f6d240cbdf0 legacy=0x7f6d240ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 -- 172.21.15.144:0/3607440102 >> v1:172.21.15.144:6789/0 conn(0x7f6d2415cef0 legacy=0x7f6d2406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d237fe700 1 -- 172.21.15.144:0/3607440102 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798515526 0 0) 0x7f6d240cc5f0 con 0x7f6d2415cef0 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 -- 172.21.15.144:0/3607440102 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2415da30 con 0x7f6d2406ba90 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d23fff700 1 --2- 172.21.15.144:0/3607440102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2406ba90 0x7f6d240af400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000bb90 tx=0x7f6d1000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59f78aedadddb33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d237fe700 1 -- 172.21.15.144:0/3607440102 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000dd10 con 0x7f6d2406ba90 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d237fe700 1 -- 172.21.15.144:0/3607440102 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1000de70 con 0x7f6d2406ba90 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.925+0000 7f6d237fe700 1 -- 172.21.15.144:0/3607440102 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10004230 con 0x7f6d2406ba90 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 -- 172.21.15.144:0/3607440102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2406ba90 msgr2=0x7f6d240af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/3607440102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2406ba90 0x7f6d240af400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000bb90 tx=0x7f6d1000bea0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 -- 172.21.15.144:0/3607440102 shutdown_connections 2023-11-28T11:51:58.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/3607440102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2406ba90 0x7f6d240af400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 -- 172.21.15.144:0/3607440102 >> 172.21.15.144:0/3607440102 conn(0x7f6d240d0000 msgr2=0x7f6d240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:58.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 -- 172.21.15.144:0/3607440102 shutdown_connections 2023-11-28T11:51:58.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.926+0000 7f6d2ae24700 1 -- 172.21.15.144:0/3607440102 wait complete. 2023-11-28T11:51:58.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.927+0000 7f6d2ae24700 1 Processor -- start 2023-11-28T11:51:58.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.927+0000 7f6d2ae24700 1 -- start start 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d2ae24700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d2406ba90 0x7f6d240c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d2ae24700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d240cbdf0 0x7f6d24158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d2ae24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 0x7f6d240b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d2ae24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b3370 con 0x7f6d2415cef0 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d2ae24700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b35a0 con 0x7f6d240cbdf0 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d2ae24700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b3880 con 0x7f6d2406ba90 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 0x7f6d240b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 0x7f6d240b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36006/0 (socket says 172.21.15.144:36006) 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 -- 172.21.15.144:0/2451019736 learned_addr learned my addr 172.21.15.144:0/2451019736 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d28bc0700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d2406ba90 0x7f6d240c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 -- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d2406ba90 msgr2=0x7f6d240c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d23fff700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d240cbdf0 0x7f6d24158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d2406ba90 0x7f6d240c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 -- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d240cbdf0 msgr2=0x7f6d24158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d240cbdf0 0x7f6d24158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 -- 172.21.15.144:0/2451019736 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1000b7f0 con 0x7f6d2415cef0 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d293c1700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 0x7f6d240b2e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c7a0 tx=0x7f6d1c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.928+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c003e10 con 0x7f6d2415cef0 2023-11-28T11:51:58.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.929+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c005070 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.929+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2415dff0 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.929+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c005420 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.929+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2415dd10 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.929+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d240cad60 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.930+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d1c019030 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.930+0000 7f6d217fa700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6d080682f0 0x7f6d0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.930+0000 7f6d28bc0700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6d080682f0 0x7f6d0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.931+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f6d1c0852d0 con 0x7f6d2415cef0 2023-11-28T11:51:58.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.931+0000 7f6d28bc0700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6d080682f0 0x7f6d0806a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d18007fd0 tx=0x7f6d18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:58.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:58.935+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6d1c051ad0 con 0x7f6d2415cef0 2023-11-28T11:51:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.159+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6d240af880 con 0x7f6d2415cef0 2023-11-28T11:51:59.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.788+0000 7f6d217fa700 1 -- 172.21.15.144:0/2451019736 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d1c0550f0 con 0x7f6d2415cef0 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6d080682f0 msgr2=0x7f6d0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6d080682f0 0x7f6d0806a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d18007fd0 tx=0x7f6d18009040 comp rx=0 tx=0).stop 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 msgr2=0x7f6d240b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 0x7f6d240b2e30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c7a0 tx=0x7f6d1c00cb60 comp rx=0 tx=0).stop 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 shutdown_connections 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d2415cef0 0x7f6d240b2e30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6d080682f0 0x7f6d0806a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d2406ba90 0x7f6d240c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 --2- 172.21.15.144:0/2451019736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d240cbdf0 0x7f6d24158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 >> 172.21.15.144:0/2451019736 conn(0x7f6d240d0000 msgr2=0x7f6d2406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:59.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 shutdown_connections 2023-11-28T11:51:59.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.791+0000 7f6d2ae24700 1 -- 172.21.15.144:0/2451019736 wait complete. 2023-11-28T11:51:59.808 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:51:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.937+0000 7fd642b3a700 1 Processor -- start 2023-11-28T11:51:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.937+0000 7fd642b3a700 1 -- start start 2023-11-28T11:51:59.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd642b3a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd63c0af060 0x7fd63c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:59.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd642b3a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:59.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd642b3a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0713c0 con 0x7fd63c06a270 2023-11-28T11:51:59.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd642b3a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c06dee0 con 0x7fd63c0af060 2023-11-28T11:51:59.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd642b3a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0716a0 con 0x7fd63c0af430 2023-11-28T11:51:59.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd63c0af060 0x7fd63c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd63c0af060 0x7fd63c070d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36010/0 (socket says 172.21.15.144:36010) 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 -- 172.21.15.144:0/3421317518 learned_addr learned my addr 172.21.15.144:0/3421317518 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 -- 172.21.15.144:0/3421317518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 msgr2=0x7fd63c06d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd63bfff700 1 --2- 172.21.15.144:0/3421317518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 --2- 172.21.15.144:0/3421317518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c06d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 -- 172.21.15.144:0/3421317518 >> v1:172.21.15.177:6789/0 conn(0x7fd63c06a270 legacy=0x7fd63c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 -- 172.21.15.144:0/3421317518 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c071980 con 0x7fd63c0af060 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd6408d6700 1 --2- 172.21.15.144:0/3421317518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c070d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd6300098f0 tx=0x7fd630009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f434e6262913c59d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd63b7fe700 1 -- 172.21.15.144:0/3421317518 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630004230 con 0x7fd63c0af060 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd63b7fe700 1 -- 172.21.15.144:0/3421317518 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd630004390 con 0x7fd63c0af060 2023-11-28T11:51:59.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.938+0000 7fd63b7fe700 1 -- 172.21.15.144:0/3421317518 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6300036a0 con 0x7fd63c0af060 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 -- 172.21.15.144:0/3421317518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 msgr2=0x7fd63c070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 --2- 172.21.15.144:0/3421317518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c070d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd6300098f0 tx=0x7fd630009c00 comp rx=0 tx=0).stop 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 -- 172.21.15.144:0/3421317518 shutdown_connections 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 --2- 172.21.15.144:0/3421317518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c070d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 --2- 172.21.15.144:0/3421317518 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c06d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 -- 172.21.15.144:0/3421317518 >> 172.21.15.144:0/3421317518 conn(0x7fd63c06b380 msgr2=0x7fd63c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 -- 172.21.15.144:0/3421317518 shutdown_connections 2023-11-28T11:51:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.939+0000 7fd642b3a700 1 -- 172.21.15.144:0/3421317518 wait complete. 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.940+0000 7fd642b3a700 1 Processor -- start 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.940+0000 7fd642b3a700 1 -- start start 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd642b3a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c06a270 0x7fd63c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd642b3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd642b3a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd642b3a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0b1480 con 0x7fd63c0af060 2023-11-28T11:51:59.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd642b3a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15ee20 con 0x7fd63c06a270 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd642b3a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15f100 con 0x7fd63c0af430 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c0af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36018/0 (socket says 172.21.15.144:36018) 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 -- 172.21.15.144:0/399048771 learned_addr learned my addr 172.21.15.144:0/399048771 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd6408d6700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c06a270 0x7fd63c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd6410d7700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:59.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 -- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 msgr2=0x7fd63c0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c0afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 -- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c06a270 msgr2=0x7fd63c156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c06a270 0x7fd63c156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd63bfff700 1 -- 172.21.15.144:0/399048771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009550 con 0x7fd63c0af060 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.941+0000 7fd6408d6700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c06a270 0x7fd63c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.942+0000 7fd63bfff700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c0af850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c7f0 tx=0x7fd62800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.942+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011800 con 0x7fd63c0af060 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.942+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628011960 con 0x7fd63c0af060 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.942+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c071f40 con 0x7fd63c0af060 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.942+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011cf0 con 0x7fd63c0af060 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.942+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c071c60 con 0x7fd63c0af060 2023-11-28T11:51:59.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.943+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c0cb670 con 0x7fd63c0af060 2023-11-28T11:51:59.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.943+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd628021070 con 0x7fd63c0af060 2023-11-28T11:51:59.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.943+0000 7fd6397fa700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200682f0 0x7fd62006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:51:59.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.944+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd628085ba0 con 0x7fd63c0af060 2023-11-28T11:51:59.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.944+0000 7fd6408d6700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200682f0 0x7fd62006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:51:59.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.944+0000 7fd6408d6700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200682f0 0x7fd62006a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd630009130 tx=0x7fd630008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:51:59.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:51:59.948+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6280523a0 con 0x7fd63c0af060 2023-11-28T11:52:00.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.130+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd63c068690 con 0x7fd63c0af060 2023-11-28T11:52:00.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.793+0000 7fd6397fa700 1 -- 172.21.15.144:0/399048771 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd62801f020 con 0x7fd63c0af060 2023-11-28T11:52:00.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200682f0 msgr2=0x7fd62006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:00.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200682f0 0x7fd62006a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd630009130 tx=0x7fd630008030 comp rx=0 tx=0).stop 2023-11-28T11:52:00.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 msgr2=0x7fd63c0af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:00.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c0af850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c7f0 tx=0x7fd62800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 shutdown_connections 2023-11-28T11:52:00.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6200682f0 0x7fd62006a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd63c0af430 0x7fd63c0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd63c06a270 0x7fd63c156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 --2- 172.21.15.144:0/399048771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c0af060 0x7fd63c0af850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 >> 172.21.15.144:0/399048771 conn(0x7fd63c06b380 msgr2=0x7fd63c0d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 shutdown_connections 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.796+0000 7fd642b3a700 1 -- 172.21.15.144:0/399048771 wait complete. 2023-11-28T11:52:00.799 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:52:00.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:52:00.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.944+0000 7f86a2551700 1 Processor -- start 2023-11-28T11:52:00.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.944+0000 7f86a2551700 1 -- start start 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a2551700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15aba0 0x7f869c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a2551700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f869c0d0140 0x7f869c0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a2551700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c15af70 con 0x7f869c0b6ae0 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a2551700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c0d0b00 con 0x7f869c0d0140 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a2551700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0cfec0 con 0x7f869c15aba0 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f869c0d0140 0x7f869c0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f869c0d0140 0x7f869c0d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36028/0 (socket says 172.21.15.144:36028) 2023-11-28T11:52:00.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 -- 172.21.15.144:0/1782400718 learned_addr learned my addr 172.21.15.144:0/1782400718 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f869bfff700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15aba0 0x7f869c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 -- 172.21.15.144:0/1782400718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15aba0 msgr2=0x7f869c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15aba0 0x7f869c0b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 -- 172.21.15.144:0/1782400718 >> v1:172.21.15.144:6789/0 conn(0x7f869c0b6ae0 legacy=0x7f869c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 -- 172.21.15.144:0/1782400718 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0cc870 con 0x7f869c0d0140 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f869bfff700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15aba0 0x7f869c0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f86a0aee700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c0d05c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8694009e50 tx=0x7f869400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab8046f837705d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.945+0000 7f869affd700 1 -- 172.21.15.144:0/1782400718 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400e070 con 0x7f869c0d0140 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f869affd700 1 -- 172.21.15.144:0/1782400718 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869400b9c0 con 0x7f869c0d0140 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f869affd700 1 -- 172.21.15.144:0/1782400718 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400bd70 con 0x7f869c0d0140 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 -- 172.21.15.144:0/1782400718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 msgr2=0x7f869c0d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:00.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c0d05c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8694009e50 tx=0x7f869400b330 comp rx=0 tx=0).stop 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 -- 172.21.15.144:0/1782400718 shutdown_connections 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c0d05c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 --2- 172.21.15.144:0/1782400718 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15aba0 0x7f869c0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 -- 172.21.15.144:0/1782400718 >> 172.21.15.144:0/1782400718 conn(0x7f869c06df70 msgr2=0x7f869c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 -- 172.21.15.144:0/1782400718 shutdown_connections 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.946+0000 7f86a2551700 1 -- 172.21.15.144:0/1782400718 wait complete. 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.947+0000 7f86a2551700 1 Processor -- start 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 -- start start 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f869c15aba0 0x7f869c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15b5a0 0x7f869c156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:00.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0b0a00 con 0x7f869c0d0140 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0cb310 con 0x7f869c15aba0 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f86a2551700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c15d880 con 0x7f869c15b5a0 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f869bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f869bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c15d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36030/0 (socket says 172.21.15.144:36030) 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.948+0000 7f869bfff700 1 -- 172.21.15.144:0/67829476 learned_addr learned my addr 172.21.15.144:0/67829476 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869b7fe700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f869c15aba0 0x7f869c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869bfff700 1 -- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15b5a0 msgr2=0x7f869c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869bfff700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15b5a0 0x7f869c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869bfff700 1 -- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f869c15aba0 msgr2=0x7f869c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869bfff700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f869c15aba0 0x7f869c0cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869bfff700 1 -- 172.21.15.144:0/67829476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694009b30 con 0x7f869c0d0140 2023-11-28T11:52:00.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f869bfff700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c15d1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f869000e810 tx=0x7f869000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000c020 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690004250 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c0cfbf0 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86900036a0 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.949+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c0b4290 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.950+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c0ccaa0 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.950+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8690003800 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.950+0000 7f8698ff9700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86880682f0 0x7f868806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.951+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f8690085810 con 0x7f869c0d0140 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.951+0000 7f869b7fe700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86880682f0 0x7f868806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.951+0000 7f869b7fe700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86880682f0 0x7f868806a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f869c1572a0 tx=0x7f868c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:00.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:00.955+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86900521d0 con 0x7f869c0d0140 2023-11-28T11:52:01.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.135+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f869c068690 con 0x7f869c0d0140 2023-11-28T11:52:01.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.799+0000 7f8698ff9700 1 -- 172.21.15.144:0/67829476 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f869000f040 con 0x7f869c0d0140 2023-11-28T11:52:01.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86880682f0 msgr2=0x7f868806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:01.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86880682f0 0x7f868806a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f869c1572a0 tx=0x7f868c009040 comp rx=0 tx=0).stop 2023-11-28T11:52:01.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 msgr2=0x7f869c15d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:01.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c15d1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f869000e810 tx=0x7f869000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:52:01.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 shutdown_connections 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f869c0d0140 0x7f869c15d1a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86880682f0 0x7f868806a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f869c15b5a0 0x7f869c156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 --2- 172.21.15.144:0/67829476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f869c15aba0 0x7f869c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 >> 172.21.15.144:0/67829476 conn(0x7f869c06df70 msgr2=0x7f869c158650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.804+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 shutdown_connections 2023-11-28T11:52:01.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:01.805+0000 7f86a2551700 1 -- 172.21.15.144:0/67829476 wait complete. 2023-11-28T11:52:01.808 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' removed 2023-11-28T11:52:01.824 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:52:01.824 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:52:01.824 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:52:01.824 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:01.824 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:52:01.824 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:52:01.825 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:52:01.825 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:01.825 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:52:01.825 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:52:01.825 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:52:01.825 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:01.825 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:52:01.826 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:52:01.826 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:52:01.826 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:01.834 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:52:01.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:52:01.835 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:52:01.835 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:52:01.837 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:52:01.837 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:52:01.838 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:52:01.838 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:52:01.877 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:52:01.877 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:01.878 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:52:01.878 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:01.880 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:52:01.880 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:01.880 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:52:01.881 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:01.881 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:52:01.961 INFO:tasks.ceph.mds.a.smithi144.stdout:starting mds.a at 2023-11-28T11:52:01.962 INFO:tasks.ceph.mds.c.smithi144.stdout:starting mds.c at 2023-11-28T11:52:01.968 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T11:52:01.968 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T11:52:02.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.014+0000 7fa0d630f700 1 Processor -- start 2023-11-28T11:52:02.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.014+0000 7fa0d630f700 1 -- start start 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d630f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa0d00cbdf0 0x7fa0d00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d630f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d00cc5f0 con 0x7fa0d006ba90 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d630f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d015d470 con 0x7fa0d015cef0 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d630f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d015d750 con 0x7fa0d00cbdf0 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d48ac700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa0d00cbdf0 0x7fa0d00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0cf7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa0d006ba90 0x7fa0d00ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52776/0 (socket says 172.21.15.144:52776) 2023-11-28T11:52:02.018 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0cf7fe700 1 -- 172.21.15.144:0/1819394525 learned_addr learned my addr 172.21.15.144:0/1819394525 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d48ac700 1 -- 172.21.15.144:0/1819394525 >> v1:172.21.15.177:6789/0 conn(0x7fa0d015cef0 legacy=0x7fa0d006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d48ac700 1 -- 172.21.15.144:0/1819394525 >> v1:172.21.15.144:6789/0 conn(0x7fa0d006ba90 legacy=0x7fa0d00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0ceffd700 1 -- 172.21.15.144:0/1819394525 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034416942 0 0) 0x7fa0d00cc5f0 con 0x7fa0d006ba90 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.015+0000 7fa0d48ac700 1 -- 172.21.15.144:0/1819394525 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d015da30 con 0x7fa0d00cbdf0 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d48ac700 1 --2- 172.21.15.144:0/1819394525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d00cbdf0 0x7fa0d00af400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009d90 tx=0x7fa0c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ca21807195db67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0ceffd700 1 -- 172.21.15.144:0/1819394525 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800e070 con 0x7fa0d00cbdf0 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0ceffd700 1 -- 172.21.15.144:0/1819394525 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800bbd0 con 0x7fa0d00cbdf0 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0ceffd700 1 -- 172.21.15.144:0/1819394525 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8012430 con 0x7fa0d00cbdf0 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 -- 172.21.15.144:0/1819394525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d00cbdf0 msgr2=0x7fa0d00af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1819394525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d00cbdf0 0x7fa0d00af400 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009d90 tx=0x7fa0c800b330 comp rx=0 tx=0).stop 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 -- 172.21.15.144:0/1819394525 shutdown_connections 2023-11-28T11:52:02.019 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1819394525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d00cbdf0 0x7fa0d00af400 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:02.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 -- 172.21.15.144:0/1819394525 >> 172.21.15.144:0/1819394525 conn(0x7fa0d00d0000 msgr2=0x7fa0d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:02.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 -- 172.21.15.144:0/1819394525 shutdown_connections 2023-11-28T11:52:02.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.016+0000 7fa0d630f700 1 -- 172.21.15.144:0/1819394525 wait complete. 2023-11-28T11:52:02.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.017+0000 7fa0d630f700 1 Processor -- start 2023-11-28T11:52:02.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 -- start start 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 0x7fa0d0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0d00cbdf0 0x7fa0d0159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0d015cef0 0x7fa0d01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d0154920 con 0x7fa0d006ba90 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c7e80 con 0x7fa0d015cef0 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0d630f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c7b80 con 0x7fa0d00cbdf0 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0cffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 0x7fa0d0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0cffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 0x7fa0d0153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36068/0 (socket says 172.21.15.144:36068) 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.018+0000 7fa0cffff700 1 -- 172.21.15.144:0/1000068191 learned_addr learned my addr 172.21.15.144:0/1000068191 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0d48ac700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0d015cef0 0x7fa0d01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:02.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cf7fe700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0d00cbdf0 0x7fa0d0159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cffff700 1 -- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0d00cbdf0 msgr2=0x7fa0d0159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cffff700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0d00cbdf0 0x7fa0d0159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cffff700 1 -- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0d015cef0 msgr2=0x7fa0d01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cffff700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0d015cef0 0x7fa0d01597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cffff700 1 -- 172.21.15.144:0/1000068191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c8009a70 con 0x7fa0d006ba90 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cf7fe700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0d00cbdf0 0x7fa0d0159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0cffff700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 0x7fa0d0153d20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000e740 tx=0x7fa0c000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00052d0 con 0x7fa0d006ba90 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0005430 con 0x7fa0d006ba90 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d015dff0 con 0x7fa0d006ba90 2023-11-28T11:52:02.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00108e0 con 0x7fa0d006ba90 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.019+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d015dd10 con 0x7fa0d006ba90 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.020+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d00cad60 con 0x7fa0d006ba90 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.020+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa0c0021070 con 0x7fa0d006ba90 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.021+0000 7fa0ccff9700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa0bc0682f0 0x7fa0bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.021+0000 7fa0cf7fe700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa0bc0682f0 0x7fa0bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.021+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa0c0087780 con 0x7fa0d006ba90 2023-11-28T11:52:02.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.021+0000 7fa0cf7fe700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa0bc0682f0 0x7fa0bc06a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0071ae0 tx=0x7fa0c4007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:02.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.025+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0c00542e0 con 0x7fa0d006ba90 2023-11-28T11:52:02.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:02.230+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa0d00c7cc0 con 0x7fa0d006ba90 2023-11-28T11:52:03.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.020+0000 7fa0ccff9700 1 -- 172.21.15.144:0/1000068191 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa0d00c7cc0 con 0x7fa0d006ba90 2023-11-28T11:52:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa0bc0682f0 msgr2=0x7fa0bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa0bc0682f0 0x7fa0bc06a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0071ae0 tx=0x7fa0c4007ce0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 msgr2=0x7fa0d0153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 0x7fa0d0153d20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000e740 tx=0x7fa0c000ea50 comp rx=0 tx=0).stop 2023-11-28T11:52:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 shutdown_connections 2023-11-28T11:52:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0d006ba90 0x7fa0d0153d20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa0bc0682f0 0x7fa0bc06a7a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0d015cef0 0x7fa0d01597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 --2- 172.21.15.144:0/1000068191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa0d00cbdf0 0x7fa0d0159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 >> 172.21.15.144:0/1000068191 conn(0x7fa0d00d0000 msgr2=0x7fa0d015c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 shutdown_connections 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.023+0000 7fa0d630f700 1 -- 172.21.15.144:0/1000068191 wait complete. 2023-11-28T11:52:03.027 INFO:teuthology.orchestra.run.smithi144.stderr: removed all blocklist entries 2023-11-28T11:52:03.039 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:52:03.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.161+0000 7fdd75c29700 1 Processor -- start 2023-11-28T11:52:03.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.161+0000 7fdd75c29700 1 -- start start 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.161+0000 7fdd75c29700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd7006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.161+0000 7fdd75c29700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7015d060 con 0x7fdd70160540 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.161+0000 7fdd75c29700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7015d340 con 0x7fdd70071040 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.161+0000 7fdd75c29700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7015d620 con 0x7fdd7006dee0 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6ffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fdd70160540 0x7fdd701609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52798/0 (socket says 172.21.15.144:52798) 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6ffff700 1 -- 172.21.15.144:0/1340830191 learned_addr learned my addr 172.21.15.144:0/1340830191 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6effd700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd7006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340929539 0 0) 0x7fdd7015d060 con 0x7fdd70160540 2023-11-28T11:52:03.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd580035b0 con 0x7fdd70160540 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637781636 0 0) 0x7fdd7015d340 con 0x7fdd70071040 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7015d060 con 0x7fdd70071040 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182159363 0 0) 0x7fdd580035b0 con 0x7fdd70160540 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 msgr2=0x7fdd7006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd7006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 >> v1:172.21.15.177:6789/0 conn(0x7fdd70071040 legacy=0x7fdd70071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7015d900 con 0x7fdd70160540 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fdd680019f0 con 0x7fdd70160540 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6effd700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd7006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 >> v1:172.21.15.144:6789/0 conn(0x7fdd70160540 legacy=0x7fdd701609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd58003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd58004100 0x7fdd58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd58004d90 0x7fdd58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7015d620 con 0x7fdd58004d90 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5800bbd0 con 0x7fdd58004100 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5800bd90 con 0x7fdd7006dee0 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd58004d90 0x7fdd58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6f7fe700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd58003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.162+0000 7fdd6effd700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd58004100 0x7fdd58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 -- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 msgr2=0x7fdd58003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd7006dee0 0x7fdd58003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 -- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd58004100 msgr2=0x7fdd58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:03.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd58004100 0x7fdd58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 -- 172.21.15.144:0/1340830191 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640094a0 con 0x7fdd58004d90 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd58004d90 0x7fdd58005200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800a7a0 tx=0x7fdd58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2d614dc7e97a8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68012600 con 0x7fdd58004d90 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd68012780 con 0x7fdd58004d90 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6e7fc700 1 -- 172.21.15.144:0/1340830191 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68012b30 con 0x7fdd58004d90 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd75c29700 1 -- 172.21.15.144:0/1340830191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd58004d90 msgr2=0x7fdd58005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd75c29700 1 --2- 172.21.15.144:0/1340830191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd58004d90 0x7fdd58005200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800a7a0 tx=0x7fdd58005880 comp rx=0 tx=0).stop 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd6ffff700 1 -- 172.21.15.144:0/1340830191 reap_dead start 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd75c29700 1 -- 172.21.15.144:0/1340830191 shutdown_connections 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd75c29700 1 -- 172.21.15.144:0/1340830191 >> 172.21.15.144:0/1340830191 conn(0x7fdd700d0000 msgr2=0x7fdd700d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:03.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd75c29700 1 -- 172.21.15.144:0/1340830191 shutdown_connections 2023-11-28T11:52:03.171 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.163+0000 7fdd75c29700 1 -- 172.21.15.144:0/1340830191 wait complete. 2023-11-28T11:52:03.171 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.165+0000 7fdd75c29700 1 Processor -- start 2023-11-28T11:52:03.171 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.165+0000 7fdd75c29700 1 -- start start 2023-11-28T11:52:03.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd7014a540 0x7fdd700c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd700c3aa0 0x7fdd700c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 0x7fdd701577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700c5170 con 0x7fdd70157380 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70157d30 con 0x7fdd7014a540 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700c44f0 con 0x7fdd700c3aa0 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 0x7fdd701577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 0x7fdd701577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36098/0 (socket says 172.21.15.144:36098) 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 -- 172.21.15.144:0/456484103 learned_addr learned my addr 172.21.15.144:0/456484103 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 -- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd700c3aa0 msgr2=0x7fdd700c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6effd700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd700c3aa0 0x7fdd700c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6f7fe700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd7014a540 0x7fdd700c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd700c3aa0 0x7fdd700c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 -- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd7014a540 msgr2=0x7fdd700c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd7014a540 0x7fdd700c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 -- 172.21.15.144:0/456484103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7006a270 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd6ffff700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 0x7fdd701577f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd68015040 tx=0x7fdd68001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68012d00 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd68012e80 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680266a0 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7015ccd0 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.166+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd700c4ab0 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.167+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7006a400 con 0x7fdd70157380 2023-11-28T11:52:03.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.168+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd68010070 con 0x7fdd70157380 2023-11-28T11:52:03.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.168+0000 7fdd57fff700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdd5c0682f0 0x7fdd5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.168+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdd68086aa0 con 0x7fdd70157380 2023-11-28T11:52:03.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.168+0000 7fdd6f7fe700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdd5c0682f0 0x7fdd5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.168+0000 7fdd6f7fe700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdd5c0682f0 0x7fdd5c06a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd600010e0 tx=0x7fdd60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.172+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd7006a400 con 0x7fdd70157380 2023-11-28T11:52:03.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.431+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdd700c52b0 con 0x7fdd70157380 2023-11-28T11:52:03.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.432+0000 7fdd57fff700 1 -- 172.21.15.144:0/456484103 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fdd700c52b0 con 0x7fdd70157380 2023-11-28T11:52:03.433 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "auth_dump": [ 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.a", 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlzuwFOBAAtN3Mhkt5bwjiPFumLWf5LQ==", 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:03.434 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.b", 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlmkasCBAAaYU+0cOXvyDnawSPLqNQBg==", 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.435 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.c", 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlyo2xAxAARq98KYTF0ViGGgRsgbiHJg==", 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.d", 2023-11-28T11:52:03.436 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVltR3sDxAAkRdNJ3ar7Vh/OwKimeVIGQ==", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.0", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAE1GVlRYZuNBAAkRLYVWzVGa/wVN7KApc9pA==", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.437 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.1", 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAH1GVlzE4mNBAA8fJmDI1X35Kb7ffo7fKFGQ==", 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.438 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.2", 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAK1GVl0BB5OBAAJXjaD/H42vlurqW5ReH8QQ==", 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.3", 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAN1GVllXIyOBAAHYoY70u/C1a0B/IZhBZ3kw==", 2023-11-28T11:52:03.439 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.4", 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAU1GVlWMw2ARAAQDzOPIY+tj6nPKoQ5d1Opw==", 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.440 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.5", 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAX1GVlwfFcABAAzJkyTNckY6Ua/4KuHrTn2A==", 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.6", 2023-11-28T11:52:03.441 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAa1GVldpcCABAAY7IyYyuR3TMLzueAnd70vA==", 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.7", 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAc1GVlnehWOxAA+UrMHeuJP4Btd7AlMXMyYA==", 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:03.442 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.0", 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVl4NRbFBAA6tNlYbw6YiCQ4X7ewAOxAw==", 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.443 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.1", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlfaJnHRAA4AdUVgWb3JW1D50Vj+quDQ==", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.2", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlxfcVGhAAEX5yipIE6u3NvW4h8fLhjw==", 2023-11-28T11:52:03.444 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.3", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlF0KoIBAAewnQNQ6UxhSPTRCTU/mgtQ==", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:03.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.admin", 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlkOuRGxAAVRU3tb5dFIW29oanM4hW6g==", 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.446 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.bootstrap-mds", 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAi1GVlH+q+NxAAHjxVxjK3KJsgQ7uvixMddQ==", 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAi1GVl/vG+NxAAfNjzT4uAzDRVGD2uAEbcgg==", 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T11:52:03.447 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.bootstrap-osd", 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAi1GVlK/q+NxAAO42ZpDbTe640DQQrxQq59Q==", 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAi1GVlKAG/NxAAIRZmrP2tmzo3FA2F8ago4g==", 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.448 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAi1GVlDAi/NxAAvWDDXcsPnHvaiiRjdeR0Hg==", 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAi1GVlFg+/NxAA2VNvCtzaV1N8jHe7u6SBkg==", 2023-11-28T11:52:03.449 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mgr.x", 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVl+4ibNBAAmnsKW1HqPNEAHCtefjHz4A==", 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile mgr", 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:03.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mgr.y", 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlU3zUMBAAYSJLQSW0CAYPWfcdqxqHbA==", 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile mgr", 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2023-11-28T11:52:03.451 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdd5c0682f0 msgr2=0x7fdd5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdd5c0682f0 0x7fdd5c06a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd600010e0 tx=0x7fdd60009040 comp rx=0 tx=0).stop 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 msgr2=0x7fdd701577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 0x7fdd701577f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdd68015040 tx=0x7fdd68001d00 comp rx=0 tx=0).stop 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 shutdown_connections 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd70157380 0x7fdd701577f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdd5c0682f0 0x7fdd5c06a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdd7014a540 0x7fdd700c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 --2- 172.21.15.144:0/456484103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd700c3aa0 0x7fdd700c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 >> 172.21.15.144:0/456484103 conn(0x7fdd700d0000 msgr2=0x7fdd7015e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 shutdown_connections 2023-11-28T11:52:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.434+0000 7fdd75c29700 1 -- 172.21.15.144:0/456484103 wait complete. 2023-11-28T11:52:03.454 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T11:52:03.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.573+0000 7f062bf9e700 1 Processor -- start 2023-11-28T11:52:03.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.574+0000 7f062bf9e700 1 -- start start 2023-11-28T11:52:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.574+0000 7f062bf9e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f062415cef0 0x7f06240af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.574+0000 7f062bf9e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f06240cbdf0 0x7f062406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.574+0000 7f062bf9e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06240cc5f0 con 0x7f062406b8c0 2023-11-28T11:52:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.574+0000 7f062bf9e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062406bd40 con 0x7f062415cef0 2023-11-28T11:52:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.574+0000 7f062bf9e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062415d470 con 0x7f06240cbdf0 2023-11-28T11:52:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f062415cef0 0x7f06240af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f062415cef0 0x7f06240af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36102/0 (socket says 172.21.15.144:36102) 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 -- 172.21.15.144:0/1040799850 learned_addr learned my addr 172.21.15.144:0/1040799850 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629539700 1 --2- 172.21.15.144:0/1040799850 >> v2:172.21.15.177:3300/0 conn(0x7f06240cbdf0 0x7f062406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 -- 172.21.15.144:0/1040799850 >> v2:172.21.15.177:3300/0 conn(0x7f06240cbdf0 msgr2=0x7f062406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 --2- 172.21.15.144:0/1040799850 >> v2:172.21.15.177:3300/0 conn(0x7f06240cbdf0 0x7f062406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 -- 172.21.15.144:0/1040799850 >> v1:172.21.15.177:6789/0 conn(0x7f062406b8c0 legacy=0x7f0624071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 -- 172.21.15.144:0/1040799850 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062415d750 con 0x7f062415cef0 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0629d3a700 1 --2- 172.21.15.144:0/1040799850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062415cef0 0x7f06240af400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f06140098f0 tx=0x7f0614009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f18f6d2039fbae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0628d38700 1 -- 172.21.15.144:0/1040799850 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614011830 con 0x7f062415cef0 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0628d38700 1 -- 172.21.15.144:0/1040799850 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614011990 con 0x7f062415cef0 2023-11-28T11:52:03.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.575+0000 7f0628d38700 1 -- 172.21.15.144:0/1040799850 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614011d20 con 0x7f062415cef0 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 -- 172.21.15.144:0/1040799850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062415cef0 msgr2=0x7f06240af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 --2- 172.21.15.144:0/1040799850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062415cef0 0x7f06240af400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f06140098f0 tx=0x7f0614009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 -- 172.21.15.144:0/1040799850 shutdown_connections 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 --2- 172.21.15.144:0/1040799850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062415cef0 0x7f06240af400 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 --2- 172.21.15.144:0/1040799850 >> v2:172.21.15.177:3300/0 conn(0x7f06240cbdf0 0x7f062406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 -- 172.21.15.144:0/1040799850 >> 172.21.15.144:0/1040799850 conn(0x7f06240d0000 msgr2=0x7f06240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 -- 172.21.15.144:0/1040799850 shutdown_connections 2023-11-28T11:52:03.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.576+0000 7f062bf9e700 1 -- 172.21.15.144:0/1040799850 wait complete. 2023-11-28T11:52:03.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.577+0000 7f062bf9e700 1 Processor -- start 2023-11-28T11:52:03.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.577+0000 7f062bf9e700 1 -- start start 2023-11-28T11:52:03.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 0x7f062414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06240cbdf0 0x7f06241f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062409d540 0x7f062415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06241f7380 con 0x7f062406b8c0 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240b4320 con 0x7f062409d540 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06241f6dc0 con 0x7f06240cbdf0 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 0x7f062414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 0x7f062414f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36108/0 (socket says 172.21.15.144:36108) 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 -- 172.21.15.144:0/3673388894 learned_addr learned my addr 172.21.15.144:0/3673388894 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062a53b700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062409d540 0x7f062415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 -- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06240cbdf0 msgr2=0x7f06241f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629539700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06240cbdf0 0x7f06241f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06240cbdf0 0x7f06241f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 -- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062409d540 msgr2=0x7f062415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062409d540 0x7f062415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 -- 172.21.15.144:0/3673388894 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009550 con 0x7f062406b8c0 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062a53b700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062409d540 0x7f062415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f0629d3a700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 0x7f062414f570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f062406c7b0 tx=0x7f0614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140044b0 con 0x7f062406b8c0 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061401f070 con 0x7f062406b8c0 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062415dd10 con 0x7f062406b8c0 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140038a0 con 0x7f062406b8c0 2023-11-28T11:52:03.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.578+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062415da30 con 0x7f062406b8c0 2023-11-28T11:52:03.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.579+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062415dff0 con 0x7f062406b8c0 2023-11-28T11:52:03.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.580+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f061402a020 con 0x7f062406b8c0 2023-11-28T11:52:03.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.580+0000 7f061a7fc700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06080682f0 0x7f060806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.580+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0614086f40 con 0x7f062406b8c0 2023-11-28T11:52:03.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.580+0000 7f0629539700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06080682f0 0x7f060806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.580+0000 7f0629539700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06080682f0 0x7f060806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0620009ab0 tx=0x7f0620008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.584+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06140b85e0 con 0x7f062406b8c0 2023-11-28T11:52:03.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.844+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f06240d0e60 con 0x7f062406b8c0 2023-11-28T11:52:03.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.849+0000 7f061a7fc700 1 -- 172.21.15.144:0/3673388894 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0614053af0 con 0x7f062406b8c0 2023-11-28T11:52:03.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06080682f0 msgr2=0x7f060806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06080682f0 0x7f060806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0620009ab0 tx=0x7f0620008040 comp rx=0 tx=0).stop 2023-11-28T11:52:03.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 msgr2=0x7f062414f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 0x7f062414f570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f062406c7b0 tx=0x7f0614004750 comp rx=0 tx=0).stop 2023-11-28T11:52:03.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 shutdown_connections 2023-11-28T11:52:03.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f062406b8c0 0x7f062414f570 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06080682f0 0x7f060806a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f062409d540 0x7f062415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.857 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 --2- 172.21.15.144:0/3673388894 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f06240cbdf0 0x7f06241f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.857 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 >> 172.21.15.144:0/3673388894 conn(0x7f06240d0000 msgr2=0x7f062406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:03.857 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 shutdown_connections 2023-11-28T11:52:03.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.851+0000 7f062bf9e700 1 -- 172.21.15.144:0/3673388894 wait complete. 2023-11-28T11:52:03.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T11:52:03.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.986+0000 7f864da2f700 1 Processor -- start 2023-11-28T11:52:03.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.986+0000 7f864da2f700 1 -- start start 2023-11-28T11:52:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f864da2f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f86480cf240 0x7f8648165450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f864da2f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864815ad60 con 0x7f86480b3090 2023-11-28T11:52:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f864da2f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86480adb90 con 0x7f86480cb620 2023-11-28T11:52:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f864da2f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86480ccbb0 con 0x7f86480cf240 2023-11-28T11:52:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8646ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f86480b3090 0x7f86480cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52828/0 (socket says 172.21.15.144:52828) 2023-11-28T11:52:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8646ffd700 1 -- 172.21.15.144:0/2762171478 learned_addr learned my addr 172.21.15.144:0/2762171478 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f86477fe700 1 --2- 172.21.15.144:0/2762171478 >> v2:172.21.15.144:3300/0 conn(0x7f86480cf240 0x7f8648165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8645ffb700 1 -- 172.21.15.144:0/2762171478 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891385802 0 0) 0x7f864815ad60 con 0x7f86480b3090 2023-11-28T11:52:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8645ffb700 1 -- 172.21.15.144:0/2762171478 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86300035b0 con 0x7f86480b3090 2023-11-28T11:52:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f86477fe700 1 -- 172.21.15.144:0/2762171478 >> v1:172.21.15.177:6789/0 conn(0x7f86480cb620 legacy=0x7f86480ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f86477fe700 1 -- 172.21.15.144:0/2762171478 >> v1:172.21.15.144:6789/0 conn(0x7f86480b3090 legacy=0x7f86480cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f86477fe700 1 -- 172.21.15.144:0/2762171478 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480ccdd0 con 0x7f86480cf240 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f86477fe700 1 --2- 172.21.15.144:0/2762171478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480cf240 0x7f8648165450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8640009d90 tx=0x7f864000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8b66d5c9edfc17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8645ffb700 1 -- 172.21.15.144:0/2762171478 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000e070 con 0x7f86480cf240 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8645ffb700 1 -- 172.21.15.144:0/2762171478 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864000bb20 con 0x7f86480cf240 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.987+0000 7f8645ffb700 1 -- 172.21.15.144:0/2762171478 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000be90 con 0x7f86480cf240 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 -- 172.21.15.144:0/2762171478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480cf240 msgr2=0x7f8648165450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 --2- 172.21.15.144:0/2762171478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480cf240 0x7f8648165450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8640009d90 tx=0x7f864000b330 comp rx=0 tx=0).stop 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 -- 172.21.15.144:0/2762171478 shutdown_connections 2023-11-28T11:52:03.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 --2- 172.21.15.144:0/2762171478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480cf240 0x7f8648165450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 -- 172.21.15.144:0/2762171478 >> 172.21.15.144:0/2762171478 conn(0x7f864809a230 msgr2=0x7f864808afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 -- 172.21.15.144:0/2762171478 shutdown_connections 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.988+0000 7f864da2f700 1 -- 172.21.15.144:0/2762171478 wait complete. 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.989+0000 7f864da2f700 1 Processor -- start 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.989+0000 7f864da2f700 1 -- start start 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f864da2f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 0x7f86480c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f864da2f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86480cb620 0x7f86480c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f864da2f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86480cf240 0x7f86480c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f864da2f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648158590 con 0x7f86480b3090 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f864da2f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86480b4c20 con 0x7f86480cb620 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f864da2f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86480c6bd0 con 0x7f86480cf240 2023-11-28T11:52:03.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 0x7f86480c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 0x7f86480c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36122/0 (socket says 172.21.15.144:36122) 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 -- 172.21.15.144:0/77455264 learned_addr learned my addr 172.21.15.144:0/77455264 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 -- 172.21.15.144:0/77455264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86480cf240 msgr2=0x7f86480c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86480cf240 0x7f86480c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 -- 172.21.15.144:0/77455264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86480cb620 msgr2=0x7f86480c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86480cb620 0x7f86480c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.990+0000 7f8646ffd700 1 -- 172.21.15.144:0/77455264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640009a70 con 0x7f86480b3090 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.991+0000 7f8646ffd700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 0x7f86480c74b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f863800c700 tx=0x7f863800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.991+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638005d70 con 0x7f86480b3090 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.991+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86380070d0 con 0x7f86480b3090 2023-11-28T11:52:03.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.991+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638007460 con 0x7f86480b3090 2023-11-28T11:52:03.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.991+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86480cd390 con 0x7f86480b3090 2023-11-28T11:52:03.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.991+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86480cd0b0 con 0x7f86480b3090 2023-11-28T11:52:03.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.992+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86480adde0 con 0x7f86480b3090 2023-11-28T11:52:04.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.992+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8638019030 con 0x7f86480b3090 2023-11-28T11:52:04.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.992+0000 7f862ffff700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86340682f0 0x7f863406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.993+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8638084e30 con 0x7f86480b3090 2023-11-28T11:52:04.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.993+0000 7f86467fc700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86340682f0 0x7f863406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.993+0000 7f86467fc700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86340682f0 0x7f863406a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f863c000f60 tx=0x7f863c007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:03.997+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8638051960 con 0x7f86480b3090 2023-11-28T11:52:04.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.253+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f86480745c0 con 0x7f86480b3090 2023-11-28T11:52:04.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.257+0000 7f862ffff700 1 -- 172.21.15.144:0/77455264 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8638054f80 con 0x7f86480b3090 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86340682f0 msgr2=0x7f863406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86340682f0 0x7f863406a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f863c000f60 tx=0x7f863c007a40 comp rx=0 tx=0).stop 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 msgr2=0x7f86480c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 0x7f86480c74b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f863800c700 tx=0x7f863800cac0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 shutdown_connections 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86480b3090 0x7f86480c74b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f86340682f0 0x7f863406a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f86480cf240 0x7f86480c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 --2- 172.21.15.144:0/77455264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86480cb620 0x7f86480c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 >> 172.21.15.144:0/77455264 conn(0x7f864809a230 msgr2=0x7f864808afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:04.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 shutdown_connections 2023-11-28T11:52:04.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.259+0000 7f864da2f700 1 -- 172.21.15.144:0/77455264 wait complete. 2023-11-28T11:52:04.273 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T11:52:04.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.392+0000 7fa65a687700 1 Processor -- start 2023-11-28T11:52:04.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.392+0000 7fa65a687700 1 -- start start 2023-11-28T11:52:04.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa65a687700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa65415cef0 0x7fa6540af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa65a687700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa6540cbdf0 0x7fa65406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa65a687700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6540cc5f0 con 0x7fa65406b8c0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa65a687700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65406bd40 con 0x7fa6540cbdf0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa65a687700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65415d470 con 0x7fa65415cef0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa658c24700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa65406b8c0 0x7fa654071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52846/0 (socket says 172.21.15.144:52846) 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa658c24700 1 -- 172.21.15.144:0/2905272369 learned_addr learned my addr 172.21.15.144:0/2905272369 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa6537fe700 1 --2- 172.21.15.144:0/2905272369 >> v2:172.21.15.177:3300/0 conn(0x7fa6540cbdf0 0x7fa65406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa653fff700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa65415cef0 0x7fa6540af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa652ffd700 1 -- 172.21.15.144:0/2905272369 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292390045 0 0) 0x7fa6540cc5f0 con 0x7fa65406b8c0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa652ffd700 1 -- 172.21.15.144:0/2905272369 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c0035b0 con 0x7fa65406b8c0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa652ffd700 1 -- 172.21.15.144:0/2905272369 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3911072810 0 0) 0x7fa63c0035b0 con 0x7fa65406b8c0 2023-11-28T11:52:04.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa6537fe700 1 -- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa65415cef0 msgr2=0x7fa6540af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa6537fe700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa65415cef0 0x7fa6540af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa6537fe700 1 -- 172.21.15.144:0/2905272369 >> v1:172.21.15.144:6789/0 conn(0x7fa65406b8c0 legacy=0x7fa654071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa6537fe700 1 -- 172.21.15.144:0/2905272369 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65415d750 con 0x7fa6540cbdf0 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.393+0000 7fa653fff700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa65415cef0 0x7fa6540af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa6537fe700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6540cbdf0 0x7fa65406b380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa644009840 tx=0x7fa644009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9adcfd3dd97a5851 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa652ffd700 1 -- 172.21.15.144:0/2905272369 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0019f0 con 0x7fa6540cbdf0 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa652ffd700 1 -- 172.21.15.144:0/2905272369 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa644004030 con 0x7fa6540cbdf0 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa652ffd700 1 -- 172.21.15.144:0/2905272369 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440043c0 con 0x7fa6540cbdf0 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 -- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6540cbdf0 msgr2=0x7fa65406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6540cbdf0 0x7fa65406b380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa644009840 tx=0x7fa644009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:04.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 -- 172.21.15.144:0/2905272369 shutdown_connections 2023-11-28T11:52:04.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa65415cef0 0x7fa6540af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 --2- 172.21.15.144:0/2905272369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6540cbdf0 0x7fa65406b380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 -- 172.21.15.144:0/2905272369 >> 172.21.15.144:0/2905272369 conn(0x7fa6540d0000 msgr2=0x7fa6540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:04.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 -- 172.21.15.144:0/2905272369 shutdown_connections 2023-11-28T11:52:04.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.394+0000 7fa65a687700 1 -- 172.21.15.144:0/2905272369 wait complete. 2023-11-28T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 Processor -- start 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 -- start start 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa65406b8c0 0x7fa654156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6540cbdf0 0x7fa6541f0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 0x7fa65406a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65409a010 con 0x7fa65415cef0 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65409a2f0 con 0x7fa65406b8c0 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.396+0000 7fa65a687700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65409a5d0 con 0x7fa6540cbdf0 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 0x7fa65406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 0x7fa65406a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36140/0 (socket says 172.21.15.144:36140) 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 -- 172.21.15.144:0/232986165 learned_addr learned my addr 172.21.15.144:0/232986165 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa6537fe700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6540cbdf0 0x7fa6541f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa653fff700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa65406b8c0 0x7fa654156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 -- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6540cbdf0 msgr2=0x7fa6541f0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6540cbdf0 0x7fa6541f0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 -- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa65406b8c0 msgr2=0x7fa654156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa65406b8c0 0x7fa654156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 -- 172.21.15.144:0/232986165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480094a0 con 0x7fa65415cef0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa653fff700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa65406b8c0 0x7fa654156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa658c24700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 0x7fa65406a270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00cca0 tx=0x7fa64c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00cea0 con 0x7fa65415cef0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6440094a0 con 0x7fa65415cef0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c00ae00 con 0x7fa65415cef0 2023-11-28T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c007310 con 0x7fa65415cef0 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.397+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65415dd10 con 0x7fa65415cef0 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.398+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65415da30 con 0x7fa65415cef0 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.398+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa64c010070 con 0x7fa65415cef0 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.399+0000 7fa650ff9700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6400682f0 0x7fa64006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.399+0000 7fa653fff700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6400682f0 0x7fa64006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.399+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa64c0854d0 con 0x7fa65415cef0 2023-11-28T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.399+0000 7fa653fff700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6400682f0 0x7fa64006a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa648009c50 tx=0x7fa648008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.403+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa64c052000 con 0x7fa65415cef0 2023-11-28T11:52:04.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.663+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa65415dff0 con 0x7fa65415cef0 2023-11-28T11:52:04.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.667+0000 7fa650ff9700 1 -- 172.21.15.144:0/232986165 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa64c0180a0 con 0x7fa65415cef0 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6400682f0 msgr2=0x7fa64006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6400682f0 0x7fa64006a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa648009c50 tx=0x7fa648008040 comp rx=0 tx=0).stop 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 msgr2=0x7fa65406a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 0x7fa65406a270 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00cca0 tx=0x7fa64c00a340 comp rx=0 tx=0).stop 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 shutdown_connections 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa65415cef0 0x7fa65406a270 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6400682f0 0x7fa64006a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa65406b8c0 0x7fa654156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 --2- 172.21.15.144:0/232986165 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6540cbdf0 0x7fa6541f0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 >> 172.21.15.144:0/232986165 conn(0x7fa6540d0000 msgr2=0x7fa65406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:04.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 shutdown_connections 2023-11-28T11:52:04.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.669+0000 7fa65a687700 1 -- 172.21.15.144:0/232986165 wait complete. 2023-11-28T11:52:04.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T11:52:04.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 Processor -- start 2023-11-28T11:52:04.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 -- start start 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e8159490 0x7f19e80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f19e80cdc80 0x7f19e80b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e80ccc00 con 0x7f19e8159860 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80ccec0 con 0x7f19e80cdc80 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19ed433700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80cd1a0 con 0x7f19e8159490 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19e77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f19e80cdc80 0x7f19e80b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.803+0000 7f19e77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f19e80cdc80 0x7f19e80b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36152/0 (socket says 172.21.15.144:36152) 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e77fe700 1 -- 172.21.15.144:0/719127475 learned_addr learned my addr 172.21.15.144:0/719127475 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e6ffd700 1 --2- 172.21.15.144:0/719127475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e8159490 0x7f19e80b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e77fe700 1 -- 172.21.15.144:0/719127475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e8159490 msgr2=0x7f19e80b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e77fe700 1 --2- 172.21.15.144:0/719127475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e8159490 0x7f19e80b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e77fe700 1 -- 172.21.15.144:0/719127475 >> v1:172.21.15.177:6789/0 conn(0x7f19e8159860 legacy=0x7f19e80c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e77fe700 1 -- 172.21.15.144:0/719127475 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80cd480 con 0x7f19e80cdc80 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e77fe700 1 --2- 172.21.15.144:0/719127475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e80b0bb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d150 tx=0x7f19e000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207b25d167f38607 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e5ffb700 1 -- 172.21.15.144:0/719127475 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0010070 con 0x7f19e80cdc80 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e5ffb700 1 -- 172.21.15.144:0/719127475 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e000dd10 con 0x7f19e80cdc80 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.804+0000 7f19e5ffb700 1 -- 172.21.15.144:0/719127475 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000b310 con 0x7f19e80cdc80 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 -- 172.21.15.144:0/719127475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 msgr2=0x7f19e80b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 --2- 172.21.15.144:0/719127475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e80b0bb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d150 tx=0x7f19e000d460 comp rx=0 tx=0).stop 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 -- 172.21.15.144:0/719127475 shutdown_connections 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 --2- 172.21.15.144:0/719127475 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e8159490 0x7f19e80b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 --2- 172.21.15.144:0/719127475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e80b0bb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 -- 172.21.15.144:0/719127475 >> 172.21.15.144:0/719127475 conn(0x7f19e80cb670 msgr2=0x7f19e80cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:04.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 -- 172.21.15.144:0/719127475 shutdown_connections 2023-11-28T11:52:04.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.805+0000 7f19ed433700 1 -- 172.21.15.144:0/719127475 wait complete. 2023-11-28T11:52:04.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.806+0000 7f19ed433700 1 Processor -- start 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.806+0000 7f19ed433700 1 -- start start 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e8159490 0x7f19e80c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e815c430 0x7f19e80c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c6cf0 con 0x7f19e80cdc80 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c6f90 con 0x7f19e8159490 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c7270 con 0x7f19e815c430 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36154/0 (socket says 172.21.15.144:36154) 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 -- 172.21.15.144:0/1435913003 learned_addr learned my addr 172.21.15.144:0/1435913003 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:04.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e67fc700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e8159490 0x7f19e80c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e77fe700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e815c430 0x7f19e80c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 -- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e815c430 msgr2=0x7f19e80c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e815c430 0x7f19e80c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 -- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e8159490 msgr2=0x7f19e80c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e8159490 0x7f19e80c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 -- 172.21.15.144:0/1435913003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e000bd40 con 0x7f19e80cdc80 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e67fc700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e8159490 0x7f19e80c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19e6ffd700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e815a800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e9b0 tx=0x7f19dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc009710 con 0x7f19e80cdc80 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00c850 con 0x7f19e80cdc80 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00cbc0 con 0x7f19e80cdc80 2023-11-28T11:52:04.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.807+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e80c9bd0 con 0x7f19e80cdc80 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.808+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e80cd760 con 0x7f19e80cdc80 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.808+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e80c9d60 con 0x7f19e80cdc80 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.809+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19dc019030 con 0x7f19e80cdc80 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.809+0000 7f19cf7fe700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19d4068320 0x7f19d406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.809+0000 7f19e67fc700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19d4068320 0x7f19d406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.809+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f19dc0858c0 con 0x7f19e80cdc80 2023-11-28T11:52:04.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.810+0000 7f19e67fc700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19d4068320 0x7f19d406a7d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19e80b4b40 tx=0x7f19d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:04.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:04.815+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19dc052470 con 0x7f19e80cdc80 2023-11-28T11:52:05.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.071+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f19e80745c0 con 0x7f19e80cdc80 2023-11-28T11:52:05.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.075+0000 7f19cf7fe700 1 -- 172.21.15.144:0/1435913003 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f19dc0170a0 con 0x7f19e80cdc80 2023-11-28T11:52:05.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19d4068320 msgr2=0x7f19d406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19d4068320 0x7f19d406a7d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f19e80b4b40 tx=0x7f19d8009040 comp rx=0 tx=0).stop 2023-11-28T11:52:05.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 msgr2=0x7f19e815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e815a800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e9b0 tx=0x7f19dc00ed70 comp rx=0 tx=0).stop 2023-11-28T11:52:05.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 shutdown_connections 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e80cdc80 0x7f19e815a800 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19d4068320 0x7f19d406a7d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e815c430 0x7f19e80c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 --2- 172.21.15.144:0/1435913003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e8159490 0x7f19e80c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 >> 172.21.15.144:0/1435913003 conn(0x7f19e80cb670 msgr2=0x7f19e806bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 shutdown_connections 2023-11-28T11:52:05.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.077+0000 7f19ed433700 1 -- 172.21.15.144:0/1435913003 wait complete. 2023-11-28T11:52:05.091 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T11:52:05.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.210+0000 7fb9b0858700 1 Processor -- start 2023-11-28T11:52:05.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.210+0000 7fb9b0858700 1 -- start start 2023-11-28T11:52:05.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9b0858700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9ac15cef0 0x7fb9ac0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9b0858700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb9ac0cbdf0 0x7fb9ac06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9b0858700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0cc5f0 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9b0858700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac06bd40 con 0x7fb9ac0cbdf0 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9b0858700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac15d470 con 0x7fb9ac15cef0 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb9ac0cbdf0 0x7fb9ac06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb9ac0cbdf0 0x7fb9ac06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36166/0 (socket says 172.21.15.144:36166) 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 -- 172.21.15.144:0/468271758 learned_addr learned my addr 172.21.15.144:0/468271758 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 -- 172.21.15.144:0/468271758 >> v2:172.21.15.177:3300/0 conn(0x7fb9ac15cef0 msgr2=0x7fb9ac0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9aa59c700 1 --2- 172.21.15.144:0/468271758 >> v2:172.21.15.177:3300/0 conn(0x7fb9ac15cef0 0x7fb9ac0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 --2- 172.21.15.144:0/468271758 >> v2:172.21.15.177:3300/0 conn(0x7fb9ac15cef0 0x7fb9ac0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 -- 172.21.15.144:0/468271758 >> v1:172.21.15.177:6789/0 conn(0x7fb9ac06b8c0 legacy=0x7fb9ac071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.211+0000 7fb9a9d9b700 1 -- 172.21.15.144:0/468271758 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac15d750 con 0x7fb9ac0cbdf0 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9a9d9b700 1 --2- 172.21.15.144:0/468271758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac0cbdf0 0x7fb9ac06b380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0098f0 tx=0x7fb99c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f6f38155ff6c42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:05.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9a959a700 1 -- 172.21.15.144:0/468271758 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004230 con 0x7fb9ac0cbdf0 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9a959a700 1 -- 172.21.15.144:0/468271758 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c004390 con 0x7fb9ac0cbdf0 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9a959a700 1 -- 172.21.15.144:0/468271758 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0036a0 con 0x7fb9ac0cbdf0 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 -- 172.21.15.144:0/468271758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac0cbdf0 msgr2=0x7fb9ac06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 --2- 172.21.15.144:0/468271758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac0cbdf0 0x7fb9ac06b380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0098f0 tx=0x7fb99c009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 -- 172.21.15.144:0/468271758 shutdown_connections 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 --2- 172.21.15.144:0/468271758 >> v2:172.21.15.177:3300/0 conn(0x7fb9ac15cef0 0x7fb9ac0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 --2- 172.21.15.144:0/468271758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac0cbdf0 0x7fb9ac06b380 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 -- 172.21.15.144:0/468271758 >> 172.21.15.144:0/468271758 conn(0x7fb9ac0d0000 msgr2=0x7fb9ac0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 -- 172.21.15.144:0/468271758 shutdown_connections 2023-11-28T11:52:05.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.212+0000 7fb9b0858700 1 -- 172.21.15.144:0/468271758 wait complete. 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.214+0000 7fb9b0858700 1 Processor -- start 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.214+0000 7fb9b0858700 1 -- start start 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9b0858700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 0x7fb9ac0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9b0858700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9ac0cbdf0 0x7fb9ac14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9b0858700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9ac15cef0 0x7fb9ac151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9b0858700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0c7420 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9b0858700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0ca8b0 con 0x7fb9ac15cef0 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9b0858700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac14e9c0 con 0x7fb9ac0cbdf0 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 0x7fb9ac0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 0x7fb9ac0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36168/0 (socket says 172.21.15.144:36168) 2023-11-28T11:52:05.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 -- 172.21.15.144:0/2768508804 learned_addr learned my addr 172.21.15.144:0/2768508804 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aad9d700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9ac15cef0 0x7fb9ac151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9a9d9b700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9ac0cbdf0 0x7fb9ac14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 -- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9ac0cbdf0 msgr2=0x7fb9ac14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9ac0cbdf0 0x7fb9ac14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 -- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9ac15cef0 msgr2=0x7fb9ac151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9ac15cef0 0x7fb9ac151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 -- 172.21.15.144:0/2768508804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c009550 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb9aa59c700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 0x7fb9ac0cad40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c820 tx=0x7fb9a000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.215+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0004220 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.216+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a0004380 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.216+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00036a0 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.216+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac15dd10 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.216+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac15da30 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.216+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac15dff0 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.217+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9a0020070 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.217+0000 7fb99affd700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb98c0682f0 0x7fb98c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.217+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb9a0085330 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.217+0000 7fb9a9d9b700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb98c0682f0 0x7fb98c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.218+0000 7fb9a9d9b700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb98c0682f0 0x7fb98c06a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009130 tx=0x7fb99c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:05.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.221+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9a0051e60 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.480+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb9ac06e490 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.484+0000 7fb99affd700 1 -- 172.21.15.144:0/2768508804 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb9a0055480 con 0x7fb9ac06b8c0 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb98c0682f0 msgr2=0x7fb98c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb98c0682f0 0x7fb98c06a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009130 tx=0x7fb99c008030 comp rx=0 tx=0).stop 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 msgr2=0x7fb9ac0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 0x7fb9ac0cad40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c820 tx=0x7fb9a000cb30 comp rx=0 tx=0).stop 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 shutdown_connections 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac06b8c0 0x7fb9ac0cad40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb98c0682f0 0x7fb98c06a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9ac15cef0 0x7fb9ac151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 --2- 172.21.15.144:0/2768508804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9ac0cbdf0 0x7fb9ac14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 >> 172.21.15.144:0/2768508804 conn(0x7fb9ac0d0000 msgr2=0x7fb9ac06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:05.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 shutdown_connections 2023-11-28T11:52:05.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.486+0000 7fb9b0858700 1 -- 172.21.15.144:0/2768508804 wait complete. 2023-11-28T11:52:05.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T11:52:05.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.619+0000 7f553da5a700 1 Processor -- start 2023-11-28T11:52:05.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.619+0000 7f553da5a700 1 -- start start 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f553da5a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5538160b10 0x7f55380cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f553da5a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f553806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f553da5a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380c9e10 con 0x7f5538160740 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f553da5a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55380ca720 con 0x7f5538160b10 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f553da5a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553806efc0 con 0x7f553806b060 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5536ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5538160740 0x7f55380cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52888/0 (socket says 172.21.15.144:52888) 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5536ffd700 1 -- 172.21.15.144:0/2643358152 learned_addr learned my addr 172.21.15.144:0/2643358152 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811133025 0 0) 0x7f55380c9e10 con 0x7f5538160740 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f55377fe700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f553806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f55367fc700 1 --2- 172.21.15.144:0/2643358152 >> v2:172.21.15.177:3300/0 conn(0x7f5538160b10 0x7f55380cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55200035b0 con 0x7f5538160740 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3192006254 0 0) 0x7f55200035b0 con 0x7f5538160740 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 msgr2=0x7f553806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f553806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 >> v2:172.21.15.177:3300/0 conn(0x7f5538160b10 msgr2=0x7f55380cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 --2- 172.21.15.144:0/2643358152 >> v2:172.21.15.177:3300/0 conn(0x7f5538160b10 0x7f55380cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380ca0d0 con 0x7f5538160740 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.620+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f552c001670 con 0x7f5538160740 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 >> v1:172.21.15.144:6789/0 conn(0x7f5538160740 legacy=0x7f55380cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f5520003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5520003f80 0x7f5520004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5520004d90 0x7f5520005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553806efc0 con 0x7f5520004d90 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552000ba10 con 0x7f5520003f80 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552000bb50 con 0x7f553806b060 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5520004d90 0x7f5520005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5536ffd700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f5520003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55367fc700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5520003f80 0x7f5520004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 -- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 msgr2=0x7f5520003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f5520003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 -- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5520003f80 msgr2=0x7f5520004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5520003f80 0x7f5520004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 -- 172.21.15.144:0/2643358152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55200035b0 con 0x7f5520004d90 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55367fc700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5520003f80 0x7f5520004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5536ffd700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f553806b060 0x7f5520003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f55377fe700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5520004d90 0x7f5520005200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5530009fc0 tx=0x7f5520005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8f59031a000bbb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553000ec30 con 0x7f5520004d90 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553000f040 con 0x7f5520004d90 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.621+0000 7f5535ffb700 1 -- 172.21.15.144:0/2643358152 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530014480 con 0x7f5520004d90 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f553da5a700 1 -- 172.21.15.144:0/2643358152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5520004d90 msgr2=0x7f5520005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f553da5a700 1 --2- 172.21.15.144:0/2643358152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5520004d90 0x7f5520005200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5530009fc0 tx=0x7f5520005880 comp rx=0 tx=0).stop 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f55377fe700 1 -- 172.21.15.144:0/2643358152 reap_dead start 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f553da5a700 1 -- 172.21.15.144:0/2643358152 shutdown_connections 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f553da5a700 1 -- 172.21.15.144:0/2643358152 >> 172.21.15.144:0/2643358152 conn(0x7f55380b12b0 msgr2=0x7f55380b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f553da5a700 1 -- 172.21.15.144:0/2643358152 shutdown_connections 2023-11-28T11:52:05.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.622+0000 7f553da5a700 1 -- 172.21.15.144:0/2643358152 wait complete. 2023-11-28T11:52:05.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.623+0000 7f553da5a700 1 Processor -- start 2023-11-28T11:52:05.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.623+0000 7f553da5a700 1 -- start start 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f553da5a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5538150be0 0x7f55380c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f553da5a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380c3830 0x7f55380ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f553da5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 0x7f55380b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f553da5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380c9750 con 0x7f55380ad810 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f553da5a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380c3ca0 con 0x7f55380c3830 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f553da5a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380ae2a0 con 0x7f5538150be0 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 0x7f55380b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 0x7f55380b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36192/0 (socket says 172.21.15.144:36192) 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 -- 172.21.15.144:0/1829681092 learned_addr learned my addr 172.21.15.144:0/1829681092 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 -- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5538150be0 msgr2=0x7f55380c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55367fc700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380c3830 0x7f55380ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f5536ffd700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5538150be0 0x7f55380c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5538150be0 0x7f55380c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 -- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380c3830 msgr2=0x7f55380ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380c3830 0x7f55380ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 -- 172.21.15.144:0/1829681092 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380cd8d0 con 0x7f55380ad810 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f5536ffd700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5538150be0 0x7f55380c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55367fc700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380c3830 0x7f55380ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.624+0000 7f55377fe700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 0x7f55380b4610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f55200046d0 tx=0x7f5530009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.625+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530007000 con 0x7f55380ad810 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.625+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5530022070 con 0x7f55380ad810 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.625+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55380ca3b0 con 0x7f55380ad810 2023-11-28T11:52:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.625+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530029480 con 0x7f55380ad810 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.625+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5538160f60 con 0x7f55380ad810 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.626+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380745c0 con 0x7f55380ad810 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.626+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f553001e430 con 0x7f55380ad810 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.627+0000 7f551ffff700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55240682f0 0x7f552406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.627+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5530024070 con 0x7f55380ad810 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.627+0000 7f5536ffd700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55240682f0 0x7f552406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.627+0000 7f5536ffd700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55240682f0 0x7f552406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f552c001eb0 tx=0x7f552c004890 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:05.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.630+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f55300547d0 con 0x7f55380ad810 2023-11-28T11:52:05.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.888+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f5538066110 con 0x7f55380ad810 2023-11-28T11:52:05.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.894+0000 7f551ffff700 1 -- 172.21.15.144:0/1829681092 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5530057df0 con 0x7f55380ad810 2023-11-28T11:52:05.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55240682f0 msgr2=0x7f552406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55240682f0 0x7f552406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f552c001eb0 tx=0x7f552c004890 comp rx=0 tx=0).stop 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 msgr2=0x7f55380b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 0x7f55380b4610 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f55200046d0 tx=0x7f5530009750 comp rx=0 tx=0).stop 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 shutdown_connections 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55380ad810 0x7f55380b4610 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f55240682f0 0x7f552406a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5538150be0 0x7f55380c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 --2- 172.21.15.144:0/1829681092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55380c3830 0x7f55380ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.896+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 >> 172.21.15.144:0/1829681092 conn(0x7f55380b12b0 msgr2=0x7f55380b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.897+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 shutdown_connections 2023-11-28T11:52:05.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:05.897+0000 7f553da5a700 1 -- 172.21.15.144:0/1829681092 wait complete. 2023-11-28T11:52:05.910 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.029+0000 7fefd19cb700 1 Processor -- start 2023-11-28T11:52:06.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.029+0000 7fefd19cb700 1 -- start start 2023-11-28T11:52:06.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefd19cb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fefcc0cdbd0 0x7fefcc071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefd19cb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fefcc071580 0x7fefcc15e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefd19cb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc071980 con 0x7fefcc154e70 2023-11-28T11:52:06.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefd19cb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc0ce050 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefd19cb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc155240 con 0x7fefcc071580 2023-11-28T11:52:06.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefca7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fefcc0cdbd0 0x7fefcc071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefca7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fefcc0cdbd0 0x7fefcc071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36198/0 (socket says 172.21.15.144:36198) 2023-11-28T11:52:06.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.030+0000 7fefca7fc700 1 -- 172.21.15.144:0/1953415746 learned_addr learned my addr 172.21.15.144:0/1953415746 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:06.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/1953415746 >> v2:172.21.15.177:3300/0 conn(0x7fefcc071580 0x7fefcc15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefca7fc700 1 -- 172.21.15.144:0/1953415746 >> v2:172.21.15.177:3300/0 conn(0x7fefcc071580 msgr2=0x7fefcc15e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefca7fc700 1 --2- 172.21.15.144:0/1953415746 >> v2:172.21.15.177:3300/0 conn(0x7fefcc071580 0x7fefcc15e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefca7fc700 1 -- 172.21.15.144:0/1953415746 >> v1:172.21.15.177:6789/0 conn(0x7fefcc154e70 legacy=0x7fefcc0cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefca7fc700 1 -- 172.21.15.144:0/1953415746 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc15b100 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/1953415746 >> v2:172.21.15.177:3300/0 conn(0x7fefcc071580 0x7fefcc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:06.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefca7fc700 1 --2- 172.21.15.144:0/1953415746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc071040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0cfb90 tx=0x7fefbc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cded77218a9df7e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefc9ffb700 1 -- 172.21.15.144:0/1953415746 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc004230 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefc9ffb700 1 -- 172.21.15.144:0/1953415746 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc004390 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefc9ffb700 1 -- 172.21.15.144:0/1953415746 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0036a0 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefd19cb700 1 -- 172.21.15.144:0/1953415746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 msgr2=0x7fefcc071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1953415746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc071040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0cfb90 tx=0x7fefbc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefd19cb700 1 -- 172.21.15.144:0/1953415746 shutdown_connections 2023-11-28T11:52:06.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1953415746 >> v2:172.21.15.177:3300/0 conn(0x7fefcc071580 0x7fefcc15e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1953415746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc071040 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.031+0000 7fefd19cb700 1 -- 172.21.15.144:0/1953415746 >> 172.21.15.144:0/1953415746 conn(0x7fefcc0d0000 msgr2=0x7fefcc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.032+0000 7fefd19cb700 1 -- 172.21.15.144:0/1953415746 shutdown_connections 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.032+0000 7fefd19cb700 1 -- 172.21.15.144:0/1953415746 wait complete. 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.033+0000 7fefd19cb700 1 Processor -- start 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.033+0000 7fefd19cb700 1 -- start start 2023-11-28T11:52:06.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefcc071580 0x7fefcc1482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefcc154e70 0x7fefcc1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc158a70 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc158d10 con 0x7fefcc154e70 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc158ff0 con 0x7fefcc071580 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36204/0 (socket says 172.21.15.144:36204) 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 -- 172.21.15.144:0/1724647032 learned_addr learned my addr 172.21.15.144:0/1724647032 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefcaffd700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefcc071580 0x7fefcc1482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefcc154e70 0x7fefcc1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 -- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefcc071580 msgr2=0x7fefcc1482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefcc071580 0x7fefcc1482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 -- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefcc154e70 msgr2=0x7fefcc1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefcc154e70 0x7fefcc1559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 -- 172.21.15.144:0/1724647032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc009550 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefcaffd700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefcc071580 0x7fefcc1482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:06.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefcc154e70 0x7fefcc1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefca7fc700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc155470 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fefcc073bb0 tx=0x7fefbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc003d80 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc01f070 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc15b6c0 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc011870 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.034+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc15b3e0 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.035+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc15b9a0 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.036+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fefbc017020 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.036+0000 7fefb3fff700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fefb80682f0 0x7fefb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.036+0000 7fefcaffd700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fefb80682f0 0x7fefb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.036+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fefbc087160 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.037+0000 7fefcaffd700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fefb80682f0 0x7fefb806a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefc0006fd0 tx=0x7fefc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.040+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fefbc053d10 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.224+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefcc0cf710 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.225+0000 7fefb3fff700 1 -- 172.21.15.144:0/1724647032 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+14257 (secure 0 0 0) 0x7fefbc057330 con 0x7fefcc0cdbd0 2023-11-28T11:52:06.227 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":30,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:03.013108+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fefb80682f0 msgr2=0x7fefb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fefb80682f0 0x7fefb806a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefc0006fd0 tx=0x7fefc0008040 comp rx=0 tx=0).stop 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 msgr2=0x7fefcc155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc155470 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fefcc073bb0 tx=0x7fefbc004750 comp rx=0 tx=0).stop 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 shutdown_connections 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fefb80682f0 0x7fefb806a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefcc154e70 0x7fefcc1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fefcc071580 0x7fefcc1482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 --2- 172.21.15.144:0/1724647032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0cdbd0 0x7fefcc155470 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 >> 172.21.15.144:0/1724647032 conn(0x7fefcc0d0000 msgr2=0x7fefcc15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:06.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 shutdown_connections 2023-11-28T11:52:06.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.227+0000 7fefd19cb700 1 -- 172.21.15.144:0/1724647032 wait complete. 2023-11-28T11:52:06.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:52:06.242 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:52:06.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.362+0000 7fae5f084700 1 Processor -- start 2023-11-28T11:52:06.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.362+0000 7fae5f084700 1 -- start start 2023-11-28T11:52:06.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae5f084700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 0x7fae580cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae5f084700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580d1950 con 0x7fae58161340 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae5f084700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580ce140 con 0x7fae5815c9b0 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae5f084700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b5800 con 0x7fae580cd7c0 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae57fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fae58161340 0x7fae580d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52924/0 (socket says 172.21.15.144:52924) 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae57fff700 1 -- 172.21.15.144:0/1895552874 learned_addr learned my addr 172.21.15.144:0/1895552874 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae5d621700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 0x7fae580cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657234198 0 0) 0x7fae580d1950 con 0x7fae58161340 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.363+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae480035b0 con 0x7fae58161340 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432430130 0 0) 0x7fae580ce140 con 0x7fae5815c9b0 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae580d1950 con 0x7fae5815c9b0 2023-11-28T11:52:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044490618 0 0) 0x7fae480035b0 con 0x7fae58161340 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 msgr2=0x7fae580cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 0x7fae580cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 >> v1:172.21.15.177:6789/0 conn(0x7fae5815c9b0 legacy=0x7fae580b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580d24f0 con 0x7fae58161340 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fae44001670 con 0x7fae58161340 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 >> v1:172.21.15.144:6789/0 conn(0x7fae58161340 legacy=0x7fae580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae48003bc0 0x7fae48003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 0x7fae48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae48004d90 0x7fae48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b5800 con 0x7fae48004d90 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4800bba0 con 0x7fae48003bc0 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4800bd60 con 0x7fae580cd7c0 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae48004d90 0x7fae48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5ce20700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae48003bc0 0x7fae48003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae57fff700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 0x7fae48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 -- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 msgr2=0x7fae48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580cd7c0 0x7fae48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 -- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae48003bc0 msgr2=0x7fae48003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae48003bc0 0x7fae48003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 -- 172.21.15.144:0/1895552874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae480035b0 con 0x7fae48004d90 2023-11-28T11:52:06.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae5d621700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae48004d90 0x7fae48005200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae50009f90 tx=0x7fae48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3dc6fcc6af5ac1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000ec30 con 0x7fae48004d90 2023-11-28T11:52:06.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5000f040 con 0x7fae48004d90 2023-11-28T11:52:06.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.364+0000 7fae577fe700 1 -- 172.21.15.144:0/1895552874 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50014480 con 0x7fae48004d90 2023-11-28T11:52:06.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5f084700 1 -- 172.21.15.144:0/1895552874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae48004d90 msgr2=0x7fae48005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5f084700 1 --2- 172.21.15.144:0/1895552874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae48004d90 0x7fae48005200 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae50009f90 tx=0x7fae48005880 comp rx=0 tx=0).stop 2023-11-28T11:52:06.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5d621700 1 -- 172.21.15.144:0/1895552874 reap_dead start 2023-11-28T11:52:06.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5f084700 1 -- 172.21.15.144:0/1895552874 shutdown_connections 2023-11-28T11:52:06.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5f084700 1 -- 172.21.15.144:0/1895552874 >> 172.21.15.144:0/1895552874 conn(0x7fae58072600 msgr2=0x7fae580cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:06.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5f084700 1 -- 172.21.15.144:0/1895552874 shutdown_connections 2023-11-28T11:52:06.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.365+0000 7fae5f084700 1 -- 172.21.15.144:0/1895552874 wait complete. 2023-11-28T11:52:06.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.367+0000 7fae5f084700 1 Processor -- start 2023-11-28T11:52:06.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.367+0000 7fae5f084700 1 -- start start 2023-11-28T11:52:06.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 0x7fae580c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580c8210 0x7fae5815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae5815e900 0x7fae580c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5815ed70 con 0x7fae58157a70 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae581f2a50 con 0x7fae5815e900 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae581f2d30 con 0x7fae580c8210 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 0x7fae580c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 0x7fae580c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36222/0 (socket says 172.21.15.144:36222) 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 -- 172.21.15.144:0/3308065422 learned_addr learned my addr 172.21.15.144:0/3308065422 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5d621700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae5815e900 0x7fae580c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae57fff700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580c8210 0x7fae5815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 -- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580c8210 msgr2=0x7fae5815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580c8210 0x7fae5815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 -- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae5815e900 msgr2=0x7fae580c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae5815e900 0x7fae580c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 -- 172.21.15.144:0/3308065422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580d1300 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5d621700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae5815e900 0x7fae580c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5ce20700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 0x7fae580c8b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00f820 tx=0x7fae4c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c005320 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c005480 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae580b5520 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c006aa0 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.368+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae580d15e0 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.370+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae580745c0 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.370+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae4c01c030 con 0x7fae58157a70 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.370+0000 7fae557fa700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fae3c0682f0 0x7fae3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.370+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fae4c085080 con 0x7fae58157a70 2023-11-28T11:52:06.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.370+0000 7fae57fff700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fae3c0682f0 0x7fae3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.371+0000 7fae57fff700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fae3c0682f0 0x7fae3c06a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fae5815f070 tx=0x7fae44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.374+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae4c051c30 con 0x7fae58157a70 2023-11-28T11:52:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.556+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae580d08a0 con 0x7fae58157a70 2023-11-28T11:52:06.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.556+0000 7fae557fa700 1 -- 172.21.15.144:0/3308065422 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae4c055250 con 0x7fae58157a70 2023-11-28T11:52:06.558 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:52:06.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fae3c0682f0 msgr2=0x7fae3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fae3c0682f0 0x7fae3c06a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fae5815f070 tx=0x7fae44009040 comp rx=0 tx=0).stop 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 msgr2=0x7fae580c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 0x7fae580c8b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00f820 tx=0x7fae4c00fbe0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 shutdown_connections 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae58157a70 0x7fae580c8b30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fae3c0682f0 0x7fae3c06a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae5815e900 0x7fae580c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 --2- 172.21.15.144:0/3308065422 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae580c8210 0x7fae5815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 >> 172.21.15.144:0/3308065422 conn(0x7fae58072600 msgr2=0x7fae5815a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:06.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 shutdown_connections 2023-11-28T11:52:06.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.558+0000 7fae5f084700 1 -- 172.21.15.144:0/3308065422 wait complete. 2023-11-28T11:52:06.572 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:52:06.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.693+0000 7f0d433cc700 1 Processor -- start 2023-11-28T11:52:06.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.694+0000 7f0d433cc700 1 -- start start 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.694+0000 7f0d433cc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 0x7f0d3c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.694+0000 7f0d433cc700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c13ff00 con 0x7f0d3c0b2e70 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.694+0000 7f0d433cc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c1404c0 con 0x7f0d3c0cd8c0 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.694+0000 7f0d433cc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c140dc0 con 0x7f0d3c140950 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d41969700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0d3c0b2e70 0x7f0d3c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52946/0 (socket says 172.21.15.144:52946) 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d41969700 1 -- 172.21.15.144:0/1080995799 learned_addr learned my addr 172.21.15.144:0/1080995799 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850042652 0 0) 0x7f0d3c13ff00 con 0x7f0d3c0b2e70 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d40967700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 0x7f0d3c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d280035b0 con 0x7f0d3c0b2e70 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872979603 0 0) 0x7f0d280035b0 con 0x7f0d3c0b2e70 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 msgr2=0x7f0d3c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 0x7f0d3c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 >> v1:172.21.15.177:6789/0 conn(0x7f0d3c0cd8c0 legacy=0x7f0d3c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c15aa80 con 0x7f0d3c0b2e70 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f0d340019f0 con 0x7f0d3c0b2e70 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 >> v1:172.21.15.144:6789/0 conn(0x7f0d3c0b2e70 legacy=0x7f0d3c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d28003a30 0x7f0d28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 0x7f0d28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d28004d90 0x7f0d28005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c140dc0 con 0x7f0d28004d90 2023-11-28T11:52:06.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2800bac0 con 0x7f0d28003a30 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2800bc50 con 0x7f0d3c140950 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d41969700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d28004d90 0x7f0d28005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d41168700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d28003a30 0x7f0d28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.695+0000 7f0d40967700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 0x7f0d28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 -- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 msgr2=0x7f0d28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c140950 0x7f0d28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 -- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d28003a30 msgr2=0x7f0d28003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d28003a30 0x7f0d28003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 -- 172.21.15.144:0/1080995799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280035b0 con 0x7f0d28004d90 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d28004d90 0x7f0d28005200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0ce800 tx=0x7f0d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c029ac32dfab61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340072d0 con 0x7f0d28004d90 2023-11-28T11:52:06.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34007430 con 0x7f0d28004d90 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d33fff700 1 -- 172.21.15.144:0/1080995799 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340077e0 con 0x7f0d28004d90 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d433cc700 1 -- 172.21.15.144:0/1080995799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d28004d90 msgr2=0x7f0d28005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d433cc700 1 --2- 172.21.15.144:0/1080995799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d28004d90 0x7f0d28005200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0ce800 tx=0x7f0d28005880 comp rx=0 tx=0).stop 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.696+0000 7f0d41969700 1 -- 172.21.15.144:0/1080995799 reap_dead start 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.697+0000 7f0d433cc700 1 -- 172.21.15.144:0/1080995799 shutdown_connections 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.697+0000 7f0d433cc700 1 -- 172.21.15.144:0/1080995799 >> 172.21.15.144:0/1080995799 conn(0x7f0d3c06aed0 msgr2=0x7f0d3c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.697+0000 7f0d433cc700 1 -- 172.21.15.144:0/1080995799 shutdown_connections 2023-11-28T11:52:06.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.697+0000 7f0d433cc700 1 -- 172.21.15.144:0/1080995799 wait complete. 2023-11-28T11:52:06.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.698+0000 7f0d433cc700 1 Processor -- start 2023-11-28T11:52:06.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.698+0000 7f0d433cc700 1 -- start start 2023-11-28T11:52:06.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 0x7f0d3c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c1f1f50 0x7f0d3c13d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3c06d070 0x7f0d3c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c0b53e0 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c06cbe0 con 0x7f0d3c06d070 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c06bfd0 con 0x7f0d3c1f1f50 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 0x7f0d3c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 0x7f0d3c13ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36242/0 (socket says 172.21.15.144:36242) 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 -- 172.21.15.144:0/3041691262 learned_addr learned my addr 172.21.15.144:0/3041691262 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41969700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3c06d070 0x7f0d3c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d40967700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c1f1f50 0x7f0d3c13d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 -- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c1f1f50 msgr2=0x7f0d3c13d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c1f1f50 0x7f0d3c13d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 -- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3c06d070 msgr2=0x7f0d3c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3c06d070 0x7f0d3c06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 -- 172.21.15.144:0/3041691262 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c0b35f0 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41969700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3c06d070 0x7f0d3c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d40967700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c1f1f50 0x7f0d3c13d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d41168700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 0x7f0d3c13ccf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00f700 tx=0x7f0d2c00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0052e0 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c005460 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3c0b3310 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.699+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c003910 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.700+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c140150 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.701+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c06cec0 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.701+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d2c003a70 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.701+0000 7f0d31ffb700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d200682f0 0x7f0d2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:06.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.701+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0d2c0859b0 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.701+0000 7f0d40967700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d200682f0 0x7f0d2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:06.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.701+0000 7f0d40967700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d200682f0 0x7f0d2006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c144320 tx=0x7f0d38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:06.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.705+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d2c052480 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.887+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d3c068690 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.887+0000 7f0d31ffb700 1 -- 172.21.15.144:0/3041691262 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d2c055aa0 con 0x7f0d3c1f1b80 2023-11-28T11:52:06.889 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:52:06.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d200682f0 msgr2=0x7f0d2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d200682f0 0x7f0d2006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c144320 tx=0x7f0d38008040 comp rx=0 tx=0).stop 2023-11-28T11:52:06.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 msgr2=0x7f0d3c13ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:06.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 0x7f0d3c13ccf0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00f700 tx=0x7f0d2c00fac0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 shutdown_connections 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d3c1f1b80 0x7f0d3c13ccf0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d200682f0 0x7f0d2006a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d3c06d070 0x7f0d3c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 --2- 172.21.15.144:0/3041691262 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d3c1f1f50 0x7f0d3c13d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 >> 172.21.15.144:0/3041691262 conn(0x7f0d3c06aed0 msgr2=0x7f0d3c0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 shutdown_connections 2023-11-28T11:52:06.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:06.889+0000 7f0d433cc700 1 -- 172.21.15.144:0/3041691262 wait complete. 2023-11-28T11:52:06.903 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:52:07.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.022+0000 7f7c426ea700 1 Processor -- start 2023-11-28T11:52:07.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.022+0000 7f7c426ea700 1 -- start start 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.023+0000 7f7c426ea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 0x7f7c3c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.023+0000 7f7c426ea700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7c3c0cf1b0 0x7f7c3c0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.023+0000 7f7c426ea700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3c0d0240 con 0x7f7c3c0cd5a0 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.023+0000 7f7c426ea700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3c0cdd60 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.023+0000 7f7c426ea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0ce040 con 0x7f7c3c15ace0 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 0x7f7c3c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c40c87700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7c3c0cf1b0 0x7f7c3c0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 0x7f7c3c0b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:48332/0 (socket says 172.21.15.144:48332) 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 -- 172.21.15.144:0/679194596 learned_addr learned my addr 172.21.15.144:0/679194596 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:07.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 -- 172.21.15.144:0/679194596 >> v2:172.21.15.177:3300/0 conn(0x7f7c3c0cf1b0 msgr2=0x7f7c3c0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 --2- 172.21.15.144:0/679194596 >> v2:172.21.15.177:3300/0 conn(0x7f7c3c0cf1b0 0x7f7c3c0d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 -- 172.21.15.144:0/679194596 >> v1:172.21.15.177:6789/0 conn(0x7f7c3c0cd5a0 legacy=0x7f7c3c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 -- 172.21.15.144:0/679194596 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0d04e0 con 0x7f7c3c15ace0 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3bfff700 1 --2- 172.21.15.144:0/679194596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 0x7f7c3c0b29c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0098f0 tx=0x7f7c2c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3183b0e6b55c6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3affd700 1 -- 172.21.15.144:0/679194596 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011830 con 0x7f7c3c15ace0 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3affd700 1 -- 172.21.15.144:0/679194596 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c011990 con 0x7f7c3c15ace0 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.024+0000 7f7c3affd700 1 -- 172.21.15.144:0/679194596 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011d20 con 0x7f7c3c15ace0 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 -- 172.21.15.144:0/679194596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 msgr2=0x7f7c3c0b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 --2- 172.21.15.144:0/679194596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 0x7f7c3c0b29c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0098f0 tx=0x7f7c2c009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 -- 172.21.15.144:0/679194596 shutdown_connections 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 --2- 172.21.15.144:0/679194596 >> v2:172.21.15.177:3300/0 conn(0x7f7c3c0cf1b0 0x7f7c3c0d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 --2- 172.21.15.144:0/679194596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c15ace0 0x7f7c3c0b29c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 -- 172.21.15.144:0/679194596 >> 172.21.15.144:0/679194596 conn(0x7f7c3c06a270 msgr2=0x7f7c3c06a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:07.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 -- 172.21.15.144:0/679194596 shutdown_connections 2023-11-28T11:52:07.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.025+0000 7f7c426ea700 1 -- 172.21.15.144:0/679194596 wait complete. 2023-11-28T11:52:07.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.026+0000 7f7c426ea700 1 Processor -- start 2023-11-28T11:52:07.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.026+0000 7f7c426ea700 1 -- start start 2023-11-28T11:52:07.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c426ea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c0cd5a0 0x7f7c3c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c426ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 0x7f7c3c155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c426ea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c3c15ace0 0x7f7c3c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c426ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c154c90 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c426ea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0b4c10 con 0x7f7c3c15ace0 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c426ea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0b4e40 con 0x7f7c3c0cd5a0 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 0x7f7c3c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 0x7f7c3c155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36258/0 (socket says 172.21.15.144:36258) 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 -- 172.21.15.144:0/4220301256 learned_addr learned my addr 172.21.15.144:0/4220301256 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c40c87700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c3c15ace0 0x7f7c3c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3bfff700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c0cd5a0 0x7f7c3c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 -- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c0cd5a0 msgr2=0x7f7c3c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c0cd5a0 0x7f7c3c154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 -- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c3c15ace0 msgr2=0x7f7c3c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c3c15ace0 0x7f7c3c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 -- 172.21.15.144:0/4220301256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c009550 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.027+0000 7f7c3b7fe700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 0x7f7c3c155080 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c4e0 tx=0x7f7c3000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.028+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c300052d0 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.028+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c30005430 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.028+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c0cda10 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.028+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c300038c0 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.028+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c0ce320 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.029+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c0d0f40 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.029+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7c30019030 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.029+0000 7f7c38ff9700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7c280682f0 0x7f7c2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.029+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7c30084ba0 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.030+0000 7f7c3bfff700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7c280682f0 0x7f7c2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.030+0000 7f7c3bfff700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7c280682f0 0x7f7c2806a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c000c00 tx=0x7f7c2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:07.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.034+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7c30051750 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.217+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7c3c068690 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.217+0000 7f7c38ff9700 1 -- 172.21.15.144:0/4220301256 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7c30017090 con 0x7f7c3c0cf1b0 2023-11-28T11:52:07.219 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:07.219 INFO:teuthology.orchestra.run.smithi144.stdout:[ 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 1, 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": "rbd" 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 2, 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": ".mgr" 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:07.220 INFO:teuthology.orchestra.run.smithi144.stdout:] 2023-11-28T11:52:07.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7c280682f0 msgr2=0x7f7c2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.221 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7c280682f0 0x7f7c2806a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c000c00 tx=0x7f7c2c008030 comp rx=0 tx=0).stop 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 msgr2=0x7f7c3c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 0x7f7c3c155080 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c4e0 tx=0x7f7c3000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 shutdown_connections 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7c280682f0 0x7f7c2806a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7c3c15ace0 0x7f7c3c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7c3c0cd5a0 0x7f7c3c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.219+0000 7f7c426ea700 1 --2- 172.21.15.144:0/4220301256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c3c0cf1b0 0x7f7c3c155080 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.220+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 >> 172.21.15.144:0/4220301256 conn(0x7f7c3c06a270 msgr2=0x7f7c3c158810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.220+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 shutdown_connections 2023-11-28T11:52:07.222 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.220+0000 7f7c426ea700 1 -- 172.21.15.144:0/4220301256 wait complete. 2023-11-28T11:52:07.233 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:52:07.234 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:52:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.356+0000 7f3178f6d700 1 Processor -- start 2023-11-28T11:52:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.356+0000 7f3178f6d700 1 -- start start 2023-11-28T11:52:07.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3178f6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31740d0140 0x7f31740d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3178f6d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317415af70 con 0x7f317415aba0 2023-11-28T11:52:07.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3178f6d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31740d0ae0 con 0x7f31740b6ae0 2023-11-28T11:52:07.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3178f6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31740cfec0 con 0x7f31740d0140 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317259c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f317415aba0 0x7f31740b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52976/0 (socket says 172.21.15.144:52976) 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317259c700 1 -- 172.21.15.144:0/2863519393 learned_addr learned my addr 172.21.15.144:0/2863519393 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3172d9d700 1 --2- 172.21.15.144:0/2863519393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31740d0140 0x7f31740d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093147468 0 0) 0x7f317415af70 con 0x7f317415aba0 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f315c0035b0 con 0x7f317415aba0 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992630024 0 0) 0x7f31740d0ae0 con 0x7f31740b6ae0 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f317415af70 con 0x7f31740b6ae0 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820841033 0 0) 0x7f315c0035b0 con 0x7f317415aba0 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3172d9d700 1 -- 172.21.15.144:0/2863519393 >> v1:172.21.15.177:6789/0 conn(0x7f31740b6ae0 legacy=0x7f31740cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3172d9d700 1 -- 172.21.15.144:0/2863519393 >> v1:172.21.15.144:6789/0 conn(0x7f317415aba0 legacy=0x7f31740b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3172d9d700 1 -- 172.21.15.144:0/2863519393 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740cc870 con 0x7f31740d0140 2023-11-28T11:52:07.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.357+0000 7f3172d9d700 1 --2- 172.21.15.144:0/2863519393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31740d0140 0x7f31740d05a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f316c009dc0 tx=0x7f316c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7a9b154ccfa54a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00e070 con 0x7f31740d0140 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c003a20 con 0x7f31740d0140 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f317159a700 1 -- 172.21.15.144:0/2863519393 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164001670 con 0x7f31740d0140 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 -- 172.21.15.144:0/2863519393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31740d0140 msgr2=0x7f31740d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 --2- 172.21.15.144:0/2863519393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31740d0140 0x7f31740d05a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f316c009dc0 tx=0x7f316c00b330 comp rx=0 tx=0).stop 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 -- 172.21.15.144:0/2863519393 shutdown_connections 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 --2- 172.21.15.144:0/2863519393 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31740d0140 0x7f31740d05a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 -- 172.21.15.144:0/2863519393 >> 172.21.15.144:0/2863519393 conn(0x7f317406df70 msgr2=0x7f317406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 -- 172.21.15.144:0/2863519393 shutdown_connections 2023-11-28T11:52:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.358+0000 7f3178f6d700 1 -- 172.21.15.144:0/2863519393 wait complete. 2023-11-28T11:52:07.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.359+0000 7f3178f6d700 1 Processor -- start 2023-11-28T11:52:07.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 -- start start 2023-11-28T11:52:07.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 0x7f3174153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317415aba0 0x7f317415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31741544c0 0x7f31740c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740ca6e0 con 0x7f31740d0140 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317415d7b0 con 0x7f31741544c0 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3178f6d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317415c900 con 0x7f317415aba0 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f317259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 0x7f3174153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f317259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 0x7f3174153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36270/0 (socket says 172.21.15.144:36270) 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f317259c700 1 -- 172.21.15.144:0/103681232 learned_addr learned my addr 172.21.15.144:0/103681232 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3171d9b700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317415aba0 0x7f317415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.365 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.360+0000 7f3172d9d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31741544c0 0x7f31740c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f317259c700 1 -- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317415aba0 msgr2=0x7f317415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f317259c700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317415aba0 0x7f317415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f317259c700 1 -- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31741544c0 msgr2=0x7f31740c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f317259c700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31741544c0 0x7f31740c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f317259c700 1 -- 172.21.15.144:0/103681232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c009a70 con 0x7f31740d0140 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f3172d9d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31741544c0 0x7f31740c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f317259c700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 0x7f3174153ed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f316400c750 tx=0x7f316400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164004c80 con 0x7f31740d0140 2023-11-28T11:52:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3164004e00 con 0x7f31740d0140 2023-11-28T11:52:07.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31740ccaa0 con 0x7f31740d0140 2023-11-28T11:52:07.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164007310 con 0x7f31740d0140 2023-11-28T11:52:07.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.361+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31740cfbf0 con 0x7f31740d0140 2023-11-28T11:52:07.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.362+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31740ccd80 con 0x7f31740d0140 2023-11-28T11:52:07.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.363+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f316401a020 con 0x7f31740d0140 2023-11-28T11:52:07.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.364+0000 7f3162ffd700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31540682f0 0x7f315406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:07.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.364+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3164085ca0 con 0x7f31740d0140 2023-11-28T11:52:07.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.364+0000 7f3171d9b700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31540682f0 0x7f315406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:07.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.364+0000 7f3171d9b700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31540682f0 0x7f315406a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31680011e0 tx=0x7f3168009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:07.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.367+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3164052780 con 0x7f31740d0140 2023-11-28T11:52:07.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.551+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f31740d0d00 con 0x7f31740d0140 2023-11-28T11:52:07.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.904+0000 7f3162ffd700 1 -- 172.21.15.144:0/103681232 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3164055da0 con 0x7f31740d0140 2023-11-28T11:52:07.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31540682f0 msgr2=0x7f315406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31540682f0 0x7f315406a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31680011e0 tx=0x7f3168009040 comp rx=0 tx=0).stop 2023-11-28T11:52:07.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 msgr2=0x7f3174153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:07.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 0x7f3174153ed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f316400c750 tx=0x7f316400ca60 comp rx=0 tx=0).stop 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 shutdown_connections 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31740d0140 0x7f3174153ed0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31540682f0 0x7f315406a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31741544c0 0x7f31740c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 --2- 172.21.15.144:0/103681232 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317415aba0 0x7f317415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 >> 172.21.15.144:0/103681232 conn(0x7f317406df70 msgr2=0x7f3174155c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 shutdown_connections 2023-11-28T11:52:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:07.908+0000 7f3178f6d700 1 -- 172.21.15.144:0/103681232 wait complete. 2023-11-28T11:52:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' created 2023-11-28T11:52:07.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:52:08.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.052+0000 7f8d4073a700 1 Processor -- start 2023-11-28T11:52:08.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.052+0000 7f8d4073a700 1 -- start start 2023-11-28T11:52:08.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.052+0000 7f8d4073a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8d38071750 0x7f8d3815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d4073a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d38155240 con 0x7f8d38154e70 2023-11-28T11:52:08.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d4073a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3815b100 con 0x7f8d380cdbd0 2023-11-28T11:52:08.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d4073a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d380ce050 con 0x7f8d38071750 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3e4d6700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8d38154e70 0x7f8d380cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52990/0 (socket says 172.21.15.144:52990) 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3e4d6700 1 -- 172.21.15.144:0/3912613208 learned_addr learned my addr 172.21.15.144:0/3912613208 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3ecd7700 1 --2- 172.21.15.144:0/3912613208 >> v2:172.21.15.177:3300/0 conn(0x7f8d38071750 0x7f8d3815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206449999 0 0) 0x7f8d38155240 con 0x7f8d38154e70 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d240035b0 con 0x7f8d38154e70 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451931518 0 0) 0x7f8d3815b100 con 0x7f8d380cdbd0 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d38155240 con 0x7f8d380cdbd0 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003768446 0 0) 0x7f8d240035b0 con 0x7f8d38154e70 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 >> v2:172.21.15.177:3300/0 conn(0x7f8d38071750 msgr2=0x7f8d3815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 --2- 172.21.15.144:0/3912613208 >> v2:172.21.15.177:3300/0 conn(0x7f8d38071750 0x7f8d3815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 >> v1:172.21.15.177:6789/0 conn(0x7f8d380cdbd0 legacy=0x7f8d38071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3815b3e0 con 0x7f8d38154e70 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f8d28001670 con 0x7f8d38154e70 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 >> v1:172.21.15.144:6789/0 conn(0x7f8d38154e70 legacy=0x7f8d380cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d24003b90 0x7f8d24003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d24004850 0x7f8d24004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24005200 0x7f8d2400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2400c030 con 0x7f8d24003b90 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2400c1a0 con 0x7f8d24005200 2023-11-28T11:52:08.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2400c360 con 0x7f8d24004850 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d24003b90 0x7f8d24003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.053+0000 7f8d3ecd7700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24005200 0x7f8d2400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3dcd5700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d24004850 0x7f8d24004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3e4d6700 1 -- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d24004850 msgr2=0x7f8d24004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d24004850 0x7f8d24004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3e4d6700 1 -- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24005200 msgr2=0x7f8d2400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24005200 0x7f8d2400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3e4d6700 1 -- 172.21.15.144:0/3912613208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d240035b0 con 0x7f8d24003b90 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3ecd7700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24005200 0x7f8d2400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3ecd7700 1 -- 172.21.15.144:0/3912613208 reap_dead start 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d24003b90 0x7f8d24003f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800c210 tx=0x7f8d24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e17d392245c74d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:08.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800cdb0 con 0x7f8d24003b90 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28005960 con 0x7f8d24003b90 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d3d4d4700 1 -- 172.21.15.144:0/3912613208 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d28005d10 con 0x7f8d24003b90 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d4073a700 1 -- 172.21.15.144:0/3912613208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d24003b90 msgr2=0x7f8d24003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d4073a700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d24003b90 0x7f8d24003f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800c210 tx=0x7f8d24005880 comp rx=0 tx=0).stop 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d4073a700 1 -- 172.21.15.144:0/3912613208 shutdown_connections 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d4073a700 1 --2- 172.21.15.144:0/3912613208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d24003b90 0x7f8d24003f80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.054+0000 7f8d4073a700 1 -- 172.21.15.144:0/3912613208 >> 172.21.15.144:0/3912613208 conn(0x7f8d380d0000 msgr2=0x7f8d380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.055+0000 7f8d4073a700 1 -- 172.21.15.144:0/3912613208 shutdown_connections 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.055+0000 7f8d4073a700 1 -- 172.21.15.144:0/3912613208 wait complete. 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 Processor -- start 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 -- start start 2023-11-28T11:52:08.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24003b90 0x7f8d3814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 0x7f8d3815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d38158ff0 0x7f8d380b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381594d0 con 0x7f8d38158080 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381597b0 con 0x7f8d24003b90 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.056+0000 7f8d4073a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38159a90 con 0x7f8d38158ff0 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 0x7f8d3815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 0x7f8d3815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36292/0 (socket says 172.21.15.144:36292) 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 -- 172.21.15.144:0/834440404 learned_addr learned my addr 172.21.15.144:0/834440404 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24003b90 0x7f8d3814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3ecd7700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d38158ff0 0x7f8d380b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 -- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d38158ff0 msgr2=0x7f8d380b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d38158ff0 0x7f8d380b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 -- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24003b90 msgr2=0x7f8d3814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24003b90 0x7f8d3814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 -- 172.21.15.144:0/834440404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3815b9a0 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24003b90 0x7f8d3814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d3dcd5700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 0x7f8d3815a570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e7b0 tx=0x7f8d3400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d340052e0 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34005440 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d340038c0 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3815b6c0 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.057+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d380af520 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.058+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d38072260 con 0x7f8d38158080 2023-11-28T11:52:08.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.059+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d34003a20 con 0x7f8d38158080 2023-11-28T11:52:08.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.059+0000 7f8d2effd700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8d1c0682f0 0x7f8d1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:08.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.059+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8d1c0682f0 0x7f8d1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:08.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.059+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f8d340852a0 con 0x7f8d38158080 2023-11-28T11:52:08.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.060+0000 7f8d3e4d6700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8d1c0682f0 0x7f8d1c06a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000c00 tx=0x7f8d2800a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:08.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.063+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d34051cf0 con 0x7f8d38158080 2023-11-28T11:52:08.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.245+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8d380b1860 con 0x7f8d38158080 2023-11-28T11:52:08.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.916+0000 7f8d2effd700 1 -- 172.21.15.144:0/834440404 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8d340190d0 con 0x7f8d38158080 2023-11-28T11:52:08.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.921+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8d1c0682f0 msgr2=0x7f8d1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.921+0000 7f8d4073a700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8d1c0682f0 0x7f8d1c06a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000c00 tx=0x7f8d2800a000 comp rx=0 tx=0).stop 2023-11-28T11:52:08.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.921+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 msgr2=0x7f8d3815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:08.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.921+0000 7f8d4073a700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 0x7f8d3815a570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400e7b0 tx=0x7f8d3400eb70 comp rx=0 tx=0).stop 2023-11-28T11:52:08.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.922+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 shutdown_connections 2023-11-28T11:52:08.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.922+0000 7f8d4073a700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8d1c0682f0 0x7f8d1c06a7a0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000c00 tx=0x7f8d2800a000 comp rx=0 tx=0).stop 2023-11-28T11:52:08.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.922+0000 7f8d4073a700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8d38158ff0 0x7f8d380b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.922+0000 7f8d4073a700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8d24003b90 0x7f8d3814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.922+0000 7f8d4073a700 1 --2- 172.21.15.144:0/834440404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d38158080 0x7f8d3815a570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:08.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.922+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 >> 172.21.15.144:0/834440404 conn(0x7f8d380d0000 msgr2=0x7f8d380b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:08.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.924+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 shutdown_connections 2023-11-28T11:52:08.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:08.925+0000 7f8d4073a700 1 -- 172.21.15.144:0/834440404 wait complete. 2023-11-28T11:52:08.930 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' created 2023-11-28T11:52:08.944 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:52:09.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.068+0000 7fd8de6ae700 1 Processor -- start 2023-11-28T11:52:09.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.068+0000 7fd8de6ae700 1 -- start start 2023-11-28T11:52:09.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8de6ae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd8d8071040 0x7fd8d8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8de6ae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd8d806dee0 0x7fd8d806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8de6ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806cdc0 0x7fd8d806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8de6ae700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d8071950 con 0x7fd8d806dee0 2023-11-28T11:52:09.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8de6ae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d815d060 con 0x7fd8d8071040 2023-11-28T11:52:09.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8de6ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d815d340 con 0x7fd8d806cdc0 2023-11-28T11:52:09.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8d77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd8d806dee0 0x7fd8d806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8d77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd8d806dee0 0x7fd8d806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36298/0 (socket says 172.21.15.144:36298) 2023-11-28T11:52:09.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.069+0000 7fd8d77fe700 1 -- 172.21.15.144:0/3694936042 learned_addr learned my addr 172.21.15.144:0/3694936042 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:09.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d7fff700 1 --2- 172.21.15.144:0/3694936042 >> v2:172.21.15.177:3300/0 conn(0x7fd8d8071040 0x7fd8d8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8dcc4b700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806cdc0 0x7fd8d806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d77fe700 1 -- 172.21.15.144:0/3694936042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806cdc0 msgr2=0x7fd8d806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d77fe700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806cdc0 0x7fd8d806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d77fe700 1 -- 172.21.15.144:0/3694936042 >> v2:172.21.15.177:3300/0 conn(0x7fd8d8071040 msgr2=0x7fd8d8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d77fe700 1 --2- 172.21.15.144:0/3694936042 >> v2:172.21.15.177:3300/0 conn(0x7fd8d8071040 0x7fd8d8071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d77fe700 1 -- 172.21.15.144:0/3694936042 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d815d620 con 0x7fd8d806dee0 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8dcc4b700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806cdc0 0x7fd8d806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d7fff700 1 --2- 172.21.15.144:0/3694936042 >> v2:172.21.15.177:3300/0 conn(0x7fd8d8071040 0x7fd8d8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d77fe700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d806dee0 0x7fd8d806e330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80d0f50 tx=0x7fd8c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfcf2b9bb56dc5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d6ffd700 1 -- 172.21.15.144:0/3694936042 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800b870 con 0x7fd8d806dee0 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d6ffd700 1 -- 172.21.15.144:0/3694936042 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800b9d0 con 0x7fd8d806dee0 2023-11-28T11:52:09.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8d6ffd700 1 -- 172.21.15.144:0/3694936042 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800bd60 con 0x7fd8d806dee0 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8de6ae700 1 -- 172.21.15.144:0/3694936042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d806dee0 msgr2=0x7fd8d806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.070+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d806dee0 0x7fd8d806e330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80d0f50 tx=0x7fd8c8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 -- 172.21.15.144:0/3694936042 shutdown_connections 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806cdc0 0x7fd8d806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/3694936042 >> v2:172.21.15.177:3300/0 conn(0x7fd8d8071040 0x7fd8d8071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/3694936042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d806dee0 0x7fd8d806e330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 -- 172.21.15.144:0/3694936042 >> 172.21.15.144:0/3694936042 conn(0x7fd8d80d0000 msgr2=0x7fd8d80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:09.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 -- 172.21.15.144:0/3694936042 shutdown_connections 2023-11-28T11:52:09.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.071+0000 7fd8de6ae700 1 -- 172.21.15.144:0/3694936042 wait complete. 2023-11-28T11:52:09.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.072+0000 7fd8de6ae700 1 Processor -- start 2023-11-28T11:52:09.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.072+0000 7fd8de6ae700 1 -- start start 2023-11-28T11:52:09.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8d806cdc0 0x7fd8d80c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806dee0 0x7fd8d80b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 0x7fd8d8152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80b4430 con 0x7fd8d8071040 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80b5220 con 0x7fd8d806cdc0 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80b4610 con 0x7fd8d806dee0 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 0x7fd8d8152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 0x7fd8d8152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36308/0 (socket says 172.21.15.144:36308) 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 -- 172.21.15.144:0/989148685 learned_addr learned my addr 172.21.15.144:0/989148685 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8d77fe700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806dee0 0x7fd8d80b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8d7fff700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8d806cdc0 0x7fd8d80c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 -- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806dee0 msgr2=0x7fd8d80b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806dee0 0x7fd8d80b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 -- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8d806cdc0 msgr2=0x7fd8d80c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8d806cdc0 0x7fd8d80c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 -- 172.21.15.144:0/989148685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8009550 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8dcc4b700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 0x7fd8d8152a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000cd10 tx=0x7fd8d000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000f480 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d000f600 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d806e870 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.073+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000f9b0 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.074+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d815d900 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.074+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d815ccd0 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.075+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd8d0009030 con 0x7fd8d8071040 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.075+0000 7fd8d4ff9700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8b80682f0 0x7fd8b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.082 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.075+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fd8d0086520 con 0x7fd8d8071040 2023-11-28T11:52:09.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.075+0000 7fd8d7fff700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8b80682f0 0x7fd8b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.076+0000 7fd8d7fff700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8b80682f0 0x7fd8b806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0006fd0 tx=0x7fd8c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.080+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd8d0052de0 con 0x7fd8d8071040 2023-11-28T11:52:09.307 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.305+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd8d80b5390 con 0x7fd8d8071040 2023-11-28T11:52:09.314 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.312+0000 7fd8d4ff9700 1 -- 172.21.15.144:0/989148685 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd8d0056400 con 0x7fd8d8071040 2023-11-28T11:52:09.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.323+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8b80682f0 msgr2=0x7fd8b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.323+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8b80682f0 0x7fd8b806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0006fd0 tx=0x7fd8c0008040 comp rx=0 tx=0).stop 2023-11-28T11:52:09.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.323+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 msgr2=0x7fd8d8152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.323+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 0x7fd8d8152a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000cd10 tx=0x7fd8d000a560 comp rx=0 tx=0).stop 2023-11-28T11:52:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 shutdown_connections 2023-11-28T11:52:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d8071040 0x7fd8d8152a10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8b80682f0 0x7fd8b806a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8d806cdc0 0x7fd8d80c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 --2- 172.21.15.144:0/989148685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8d806dee0 0x7fd8d80b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 >> 172.21.15.144:0/989148685 conn(0x7fd8d80d0000 msgr2=0x7fd8d806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:09.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 shutdown_connections 2023-11-28T11:52:09.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.324+0000 7fd8de6ae700 1 -- 172.21.15.144:0/989148685 wait complete. 2023-11-28T11:52:09.331 INFO:teuthology.orchestra.run.smithi144.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T11:52:09.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:09.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.464+0000 7f28337f2700 1 Processor -- start 2023-11-28T11:52:09.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.464+0000 7f28337f2700 1 -- start start 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f28337f2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f282c0d0a00 0x7f282c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f28337f2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c06b420 con 0x7f282c0ce040 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f28337f2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c06b700 con 0x7f282c0b2930 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f28337f2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282c06ff60 con 0x7f282c0d0a00 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f2830d8d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f282c0d0a00 0x7f282c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f283158e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f282c0ce040 0x7f282c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53040/0 (socket says 172.21.15.144:53040) 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.465+0000 7f283158e700 1 -- 172.21.15.144:0/3624221376 learned_addr learned my addr 172.21.15.144:0/3624221376 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2830d8d700 1 -- 172.21.15.144:0/3624221376 >> v1:172.21.15.177:6789/0 conn(0x7f282c0b2930 legacy=0x7f282c06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2830d8d700 1 -- 172.21.15.144:0/3624221376 >> v1:172.21.15.144:6789/0 conn(0x7f282c0ce040 legacy=0x7f282c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2823fff700 1 -- 172.21.15.144:0/3624221376 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800422400 0 0) 0x7f282c06b420 con 0x7f282c0ce040 2023-11-28T11:52:09.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2830d8d700 1 -- 172.21.15.144:0/3624221376 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c06b9e0 con 0x7f282c0d0a00 2023-11-28T11:52:09.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2830d8d700 1 --2- 172.21.15.144:0/3624221376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0d0a00 0x7f282c15df10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f281c009840 tx=0x7f281c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd9ba30c13a5831 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2823fff700 1 -- 172.21.15.144:0/3624221376 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c004230 con 0x7f282c0d0a00 2023-11-28T11:52:09.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2823fff700 1 -- 172.21.15.144:0/3624221376 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c004390 con 0x7f282c0d0a00 2023-11-28T11:52:09.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f2823fff700 1 -- 172.21.15.144:0/3624221376 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c0036a0 con 0x7f282c0d0a00 2023-11-28T11:52:09.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f28337f2700 1 -- 172.21.15.144:0/3624221376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0d0a00 msgr2=0x7f282c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.466+0000 7f28337f2700 1 --2- 172.21.15.144:0/3624221376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0d0a00 0x7f282c15df10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f281c009840 tx=0x7f281c009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:09.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.467+0000 7f28337f2700 1 -- 172.21.15.144:0/3624221376 shutdown_connections 2023-11-28T11:52:09.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.467+0000 7f28337f2700 1 --2- 172.21.15.144:0/3624221376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0d0a00 0x7f282c15df10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.467+0000 7f28337f2700 1 -- 172.21.15.144:0/3624221376 >> 172.21.15.144:0/3624221376 conn(0x7f282c06aed0 msgr2=0x7f282c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:09.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.467+0000 7f28337f2700 1 -- 172.21.15.144:0/3624221376 shutdown_connections 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.467+0000 7f28337f2700 1 -- 172.21.15.144:0/3624221376 wait complete. 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 Processor -- start 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 -- start start 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f282c0b2930 0x7f282c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 0x7f282c0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f282c0d0a00 0x7f282c0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c0c7870 con 0x7f282c0ce040 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.468+0000 7f28337f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c0c7350 con 0x7f282c0b2930 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f28337f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c0c7590 con 0x7f282c0d0a00 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 0x7f282c0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 0x7f282c0c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36336/0 (socket says 172.21.15.144:36336) 2023-11-28T11:52:09.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 -- 172.21.15.144:0/921101934 learned_addr learned my addr 172.21.15.144:0/921101934 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f283158e700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f282c0b2930 0x7f282c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2831d8f700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f282c0d0a00 0x7f282c0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 -- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f282c0d0a00 msgr2=0x7f282c0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f282c0d0a00 0x7f282c0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 -- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f282c0b2930 msgr2=0x7f282c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f282c0b2930 0x7f282c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 -- 172.21.15.144:0/921101934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0094a0 con 0x7f282c0ce040 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2830d8d700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 0x7f282c0c68d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f281c0090b0 tx=0x7f281c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c011830 con 0x7f282c0ce040 2023-11-28T11:52:09.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c011990 con 0x7f282c0ce040 2023-11-28T11:52:09.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c06bfa0 con 0x7f282c0ce040 2023-11-28T11:52:09.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c011d40 con 0x7f282c0ce040 2023-11-28T11:52:09.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.469+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c06bcc0 con 0x7f282c0ce040 2023-11-28T11:52:09.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.470+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c0b2db0 con 0x7f282c0ce040 2023-11-28T11:52:09.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.471+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f281c017020 con 0x7f282c0ce040 2023-11-28T11:52:09.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.471+0000 7f2821ffb700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f28100682f0 0x7f281006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.471+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f281c087590 con 0x7f282c0ce040 2023-11-28T11:52:09.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.471+0000 7f283158e700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f28100682f0 0x7f281006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.472+0000 7f283158e700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f28100682f0 0x7f281006a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2828007fd0 tx=0x7f2828009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.475+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f281c053ca0 con 0x7f282c0ce040 2023-11-28T11:52:09.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.658+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f282c0c8fd0 con 0x7f282c0ce040 2023-11-28T11:52:09.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.659+0000 7f2821ffb700 1 -- 172.21.15.144:0/921101934 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+17982 (secure 0 0 0) 0x7f282c0c8fd0 con 0x7f282c0ce040 2023-11-28T11:52:09.661 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:09.661 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":33,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:09.307587+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f28100682f0 msgr2=0x7f281006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f28100682f0 0x7f281006a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2828007fd0 tx=0x7f2828009040 comp rx=0 tx=0).stop 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 msgr2=0x7f282c0c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 0x7f282c0c68d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f281c0090b0 tx=0x7f281c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 shutdown_connections 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f28100682f0 0x7f281006a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f282c0d0a00 0x7f282c0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f282c0b2930 0x7f282c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 --2- 172.21.15.144:0/921101934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f282c0ce040 0x7f282c0c68d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 >> 172.21.15.144:0/921101934 conn(0x7f282c06aed0 msgr2=0x7f282c15cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:09.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 shutdown_connections 2023-11-28T11:52:09.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.662+0000 7f28337f2700 1 -- 172.21.15.144:0/921101934 wait complete. 2023-11-28T11:52:09.676 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:09.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.795+0000 7f232f13e700 1 Processor -- start 2023-11-28T11:52:09.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.795+0000 7f232f13e700 1 -- start start 2023-11-28T11:52:09.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.795+0000 7f232f13e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23280cd8c0 0x7f2328140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232f13e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f23280b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232f13e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232813ff00 con 0x7f23280b2e70 2023-11-28T11:52:09.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232f13e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23281404c0 con 0x7f23280cd8c0 2023-11-28T11:52:09.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232f13e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232815aa80 con 0x7f23280cdc90 2023-11-28T11:52:09.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23280cd8c0 0x7f2328140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23280cd8c0 0x7f2328140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36342/0 (socket says 172.21.15.144:36342) 2023-11-28T11:52:09.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 -- 172.21.15.144:0/360792383 learned_addr learned my addr 172.21.15.144:0/360792383 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:09.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 -- 172.21.15.144:0/360792383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 msgr2=0x7f23280b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:09.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 --2- 172.21.15.144:0/360792383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f23280b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 -- 172.21.15.144:0/360792383 >> v1:172.21.15.177:6789/0 conn(0x7f23280b2e70 legacy=0x7f232806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 -- 172.21.15.144:0/360792383 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280b3310 con 0x7f23280cd8c0 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f232ceda700 1 --2- 172.21.15.144:0/360792383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f2328140950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f231c006fd0 tx=0x7f231c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17465dcdbc57e113 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f23277fe700 1 -- 172.21.15.144:0/360792383 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c005e30 con 0x7f23280cd8c0 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f23277fe700 1 -- 172.21.15.144:0/360792383 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c006030 con 0x7f23280cd8c0 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.796+0000 7f23277fe700 1 -- 172.21.15.144:0/360792383 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c0063c0 con 0x7f23280cd8c0 2023-11-28T11:52:09.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 -- 172.21.15.144:0/360792383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 msgr2=0x7f2328140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 --2- 172.21.15.144:0/360792383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f2328140950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f231c006fd0 tx=0x7f231c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 -- 172.21.15.144:0/360792383 shutdown_connections 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 --2- 172.21.15.144:0/360792383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f2328140950 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 --2- 172.21.15.144:0/360792383 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f23280b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 -- 172.21.15.144:0/360792383 >> 172.21.15.144:0/360792383 conn(0x7f232806aed0 msgr2=0x7f232806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 -- 172.21.15.144:0/360792383 shutdown_connections 2023-11-28T11:52:09.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.797+0000 7f232f13e700 1 -- 172.21.15.144:0/360792383 wait complete. 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.798+0000 7f232f13e700 1 Processor -- start 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.798+0000 7f232f13e700 1 -- start start 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232f13e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280b2e70 0x7f232813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232f13e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f232813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232f13e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f232806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232f13e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b3da0 con 0x7f23280cd8c0 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232f13e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b4080 con 0x7f23280b2e70 2023-11-28T11:52:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232f13e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b4360 con 0x7f23280cdc90 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f232813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f232813d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36350/0 (socket says 172.21.15.144:36350) 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 -- 172.21.15.144:0/946809712 learned_addr learned my addr 172.21.15.144:0/946809712 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232ceda700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280b2e70 0x7f232813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f232d6db700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f232806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 -- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 msgr2=0x7f232806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f232806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 -- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280b2e70 msgr2=0x7f232813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280b2e70 0x7f232813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.799+0000 7f2327fff700 1 -- 172.21.15.144:0/946809712 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c009500 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.800+0000 7f2327fff700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f232813d230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f231400b450 tx=0x7f231400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.800+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314011830 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.800+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2314011990 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.800+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314011d40 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.800+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2328140150 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.800+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23280b35f0 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.801+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23280ce100 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.801+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2314021070 con 0x7f23280cd8c0 2023-11-28T11:52:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.801+0000 7f23257fa700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f230c0682f0 0x7f230c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:09.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.801+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2314085b50 con 0x7f23280cd8c0 2023-11-28T11:52:09.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.801+0000 7f232ceda700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f230c0682f0 0x7f230c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:09.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.802+0000 7f232ceda700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f230c0682f0 0x7f230c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f231c013040 tx=0x7f231c008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:09.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.805+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f23140523b0 con 0x7f23280cd8c0 2023-11-28T11:52:09.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.992+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2328068690 con 0x7f23280cd8c0 2023-11-28T11:52:09.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.994+0000 7f23257fa700 1 -- 172.21.15.144:0/946809712 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+17982 (secure 0 0 0) 0x7f2314015090 con 0x7f23280cd8c0 2023-11-28T11:52:09.996 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:09.996 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":33,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:09.307587+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f230c0682f0 msgr2=0x7f230c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f230c0682f0 0x7f230c06a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f231c013040 tx=0x7f231c008070 comp rx=0 tx=0).stop 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 msgr2=0x7f232813d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f232813d230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f231400b450 tx=0x7f231400b810 comp rx=0 tx=0).stop 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 shutdown_connections 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f230c0682f0 0x7f230c06a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280cdc90 0x7f232806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280b2e70 0x7f232813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 --2- 172.21.15.144:0/946809712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23280cd8c0 0x7f232813d230 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:09.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 >> 172.21.15.144:0/946809712 conn(0x7f232806aed0 msgr2=0x7f232813f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:09.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 shutdown_connections 2023-11-28T11:52:09.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:09.996+0000 7f232f13e700 1 -- 172.21.15.144:0/946809712 wait complete. 2023-11-28T11:52:10.010 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:52:10.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 Processor -- start 2023-11-28T11:52:10.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 -- start start 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f39b80d0a00 0x7f39b815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b806b420 con 0x7f39b80ce040 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b80b32f0 con 0x7f39b80d0a00 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.132+0000 7f39bfd9f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b806ff60 con 0x7f39b80b2930 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f39b80d0a00 0x7f39b815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f39b80d0a00 0x7f39b815df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36358/0 (socket says 172.21.15.144:36358) 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 -- 172.21.15.144:0/1512538622 learned_addr learned my addr 172.21.15.144:0/1512538622 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39be33c700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:10.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 -- 172.21.15.144:0/1512538622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 msgr2=0x7f39b80b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 -- 172.21.15.144:0/1512538622 >> v1:172.21.15.177:6789/0 conn(0x7f39b80ce040 legacy=0x7f39b806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 -- 172.21.15.144:0/1512538622 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b806b700 con 0x7f39b80d0a00 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39be33c700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bd33a700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80d0a00 0x7f39b815df10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f39b40098f0 tx=0x7f39b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1613b2f3f644eed4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bcb39700 1 -- 172.21.15.144:0/1512538622 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400b8a0 con 0x7f39b80d0a00 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bcb39700 1 -- 172.21.15.144:0/1512538622 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b400ba00 con 0x7f39b80d0a00 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.133+0000 7f39bcb39700 1 -- 172.21.15.144:0/1512538622 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400bd90 con 0x7f39b80d0a00 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 -- 172.21.15.144:0/1512538622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80d0a00 msgr2=0x7f39b815df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80d0a00 0x7f39b815df10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f39b40098f0 tx=0x7f39b4009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:10.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 -- 172.21.15.144:0/1512538622 shutdown_connections 2023-11-28T11:52:10.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:10.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/1512538622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80d0a00 0x7f39b815df10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:10.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 -- 172.21.15.144:0/1512538622 >> 172.21.15.144:0/1512538622 conn(0x7f39b806aed0 msgr2=0x7f39b806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:10.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 -- 172.21.15.144:0/1512538622 shutdown_connections 2023-11-28T11:52:10.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.134+0000 7f39bfd9f700 1 -- 172.21.15.144:0/1512538622 wait complete. 2023-11-28T11:52:10.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.135+0000 7f39bfd9f700 1 Processor -- start 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.135+0000 7f39bfd9f700 1 -- start start 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bfd9f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bfd9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 0x7f39b80ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bfd9f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39b80d0a00 0x7f39b80d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bfd9f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b80c5910 con 0x7f39b80ce040 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bfd9f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b81f2030 con 0x7f39b80d0a00 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bfd9f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b80c6270 con 0x7f39b80b2930 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 0x7f39b80ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 0x7f39b80ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36364/0 (socket says 172.21.15.144:36364) 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 -- 172.21.15.144:0/3263110680 learned_addr learned my addr 172.21.15.144:0/3263110680 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:10.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bdb3b700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39be33c700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39b80d0a00 0x7f39b80d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 -- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 msgr2=0x7f39b80c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 -- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39b80d0a00 msgr2=0x7f39b80d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39b80d0a00 0x7f39b80d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 -- 172.21.15.144:0/3263110680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4009550 con 0x7f39b80ce040 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39be33c700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39b80d0a00 0x7f39b80d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39bd33a700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 0x7f39b80ca760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009c40 tx=0x7f39b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.136+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40036a0 con 0x7f39b80ce040 2023-11-28T11:52:10.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.137+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4003800 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.137+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b806bcc0 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.137+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003bb0 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.137+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b806b9e0 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.137+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b806bfa0 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.138+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39b402a020 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.138+0000 7f39ae7fc700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f399c0682f0 0x7f399c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.138+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39b40873d0 con 0x7f39b80ce040 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.138+0000 7f39bdb3b700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f399c0682f0 0x7f399c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:10.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.139+0000 7f39bdb3b700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f399c0682f0 0x7f399c06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f39a8000fb0 tx=0x7f39a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:10.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.143+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39b4053c30 con 0x7f39b80ce040 2023-11-28T11:52:10.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:10.370+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f39b80c5a50 con 0x7f39b80ce040 2023-11-28T11:52:11.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.327+0000 7f39ae7fc700 1 -- 172.21.15.144:0/3263110680 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f39b80c5a50 con 0x7f39b80ce040 2023-11-28T11:52:11.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f399c0682f0 msgr2=0x7f399c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:11.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f399c0682f0 0x7f399c06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f39a8000fb0 tx=0x7f39a8009040 comp rx=0 tx=0).stop 2023-11-28T11:52:11.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 msgr2=0x7f39b80ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:11.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 0x7f39b80ca760 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009c40 tx=0x7f39b4004750 comp rx=0 tx=0).stop 2023-11-28T11:52:11.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 shutdown_connections 2023-11-28T11:52:11.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f399c0682f0 0x7f399c06a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39b80d0a00 0x7f39b80d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39b80b2930 0x7f39b80c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 --2- 172.21.15.144:0/3263110680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39b80ce040 0x7f39b80ca760 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 >> 172.21.15.144:0/3263110680 conn(0x7f39b806aed0 msgr2=0x7f39b80cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:11.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 shutdown_connections 2023-11-28T11:52:11.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.330+0000 7f39bfd9f700 1 -- 172.21.15.144:0/3263110680 wait complete. 2023-11-28T11:52:11.344 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:52:11.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:52:11.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.470+0000 7fa81a303700 1 Processor -- start 2023-11-28T11:52:11.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.470+0000 7fa81a303700 1 -- start start 2023-11-28T11:52:11.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa81a303700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa814071040 0x7fa814071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:11.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa81a303700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa814071950 con 0x7fa81406dee0 2023-11-28T11:52:11.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa81a303700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81415d060 con 0x7fa814160540 2023-11-28T11:52:11.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa81a303700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81415d340 con 0x7fa814071040 2023-11-28T11:52:11.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa813fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa814071040 0x7fa814071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:11.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa8137fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa81406dee0 0x7fa81406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53082/0 (socket says 172.21.15.144:53082) 2023-11-28T11:52:11.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa8137fe700 1 -- 172.21.15.144:0/1326026119 learned_addr learned my addr 172.21.15.144:0/1326026119 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:11.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa813fff700 1 -- 172.21.15.144:0/1326026119 >> v1:172.21.15.177:6789/0 conn(0x7fa814160540 legacy=0x7fa814160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:11.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa813fff700 1 -- 172.21.15.144:0/1326026119 >> v1:172.21.15.144:6789/0 conn(0x7fa81406dee0 legacy=0x7fa81406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa812ffd700 1 -- 172.21.15.144:0/1326026119 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658399510 0 0) 0x7fa814071950 con 0x7fa81406dee0 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa813fff700 1 -- 172.21.15.144:0/1326026119 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81415d620 con 0x7fa814071040 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.471+0000 7fa813fff700 1 --2- 172.21.15.144:0/1326026119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa814071040 0x7fa814071410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa808009840 tx=0x7fa808009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4d8273422c53ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa812ffd700 1 -- 172.21.15.144:0/1326026119 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808004230 con 0x7fa814071040 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa812ffd700 1 -- 172.21.15.144:0/1326026119 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808004390 con 0x7fa814071040 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa812ffd700 1 -- 172.21.15.144:0/1326026119 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8080036a0 con 0x7fa814071040 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 -- 172.21.15.144:0/1326026119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa814071040 msgr2=0x7fa814071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:11.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 --2- 172.21.15.144:0/1326026119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa814071040 0x7fa814071410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa808009840 tx=0x7fa808009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 -- 172.21.15.144:0/1326026119 shutdown_connections 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 --2- 172.21.15.144:0/1326026119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa814071040 0x7fa814071410 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 -- 172.21.15.144:0/1326026119 >> 172.21.15.144:0/1326026119 conn(0x7fa8140d0000 msgr2=0x7fa8140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 -- 172.21.15.144:0/1326026119 shutdown_connections 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.472+0000 7fa81a303700 1 -- 172.21.15.144:0/1326026119 wait complete. 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.473+0000 7fa81a303700 1 Processor -- start 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.473+0000 7fa81a303700 1 -- start start 2023-11-28T11:52:11.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa81a303700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 0x7fa8141f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa81a303700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa814071040 0x7fa8140c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa81a303700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa814160540 0x7fa8141578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa81a303700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141f2d70 con 0x7fa81406dee0 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa81a303700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140c4850 con 0x7fa814071040 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa81a303700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140c3c40 con 0x7fa814160540 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa813fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 0x7fa8141f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa813fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 0x7fa8141f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36376/0 (socket says 172.21.15.144:36376) 2023-11-28T11:52:11.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa813fff700 1 -- 172.21.15.144:0/1185292096 learned_addr learned my addr 172.21.15.144:0/1185292096 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:11.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa8137fe700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa814071040 0x7fa8140c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:11.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.474+0000 7fa8188a0700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa814160540 0x7fa8141578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:11.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa813fff700 1 -- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa814160540 msgr2=0x7fa8141578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:11.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa813fff700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa814160540 0x7fa8141578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa813fff700 1 -- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa814071040 msgr2=0x7fa8140c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:11.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa813fff700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa814071040 0x7fa8140c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa813fff700 1 -- 172.21.15.144:0/1185292096 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080094a0 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa813fff700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 0x7fa8141f2830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa8080090b0 tx=0x7fa8080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808011630 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa808011790 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81415ccd0 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808011b40 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.475+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81415d900 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.476+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81406a270 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.477+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa808017020 con 0x7fa81406dee0 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.477+0000 7fa810ff9700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa800068320 0x7fa80006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:11.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.477+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa808087a00 con 0x7fa81406dee0 2023-11-28T11:52:11.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.477+0000 7fa8137fe700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa800068320 0x7fa80006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:11.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.477+0000 7fa8137fe700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa800068320 0x7fa80006a7d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa81406af10 tx=0x7fa804007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:11.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.481+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa808054300 con 0x7fa81406dee0 2023-11-28T11:52:11.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:11.710+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa81415c4f0 con 0x7fa81406dee0 2023-11-28T11:52:12.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.329+0000 7fa810ff9700 1 -- 172.21.15.144:0/1185292096 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa808057920 con 0x7fa81406dee0 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa800068320 msgr2=0x7fa80006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa800068320 0x7fa80006a7d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa81406af10 tx=0x7fa804007c90 comp rx=0 tx=0).stop 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 msgr2=0x7fa8141f2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 0x7fa8141f2830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa8080090b0 tx=0x7fa8080047c0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 shutdown_connections 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406dee0 0x7fa8141f2830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa800068320 0x7fa80006a7d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa814160540 0x7fa8141578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 --2- 172.21.15.144:0/1185292096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa814071040 0x7fa8140c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 >> 172.21.15.144:0/1185292096 conn(0x7fa8140d0000 msgr2=0x7fa81415f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:12.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 shutdown_connections 2023-11-28T11:52:12.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.332+0000 7fa81a303700 1 -- 172.21.15.144:0/1185292096 wait complete. 2023-11-28T11:52:12.347 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:52:12.347 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:12.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.470+0000 7f4d812e0700 1 Processor -- start 2023-11-28T11:52:12.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.470+0000 7f4d812e0700 1 -- start start 2023-11-28T11:52:12.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d812e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d7c0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d812e0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d7c0ccc40 0x7f4d7c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d812e0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c0cf5e0 con 0x7f4d7c15ca50 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d812e0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cd470 con 0x7f4d7c0ccc40 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d812e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cecb0 con 0x7f4d7c06ba90 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d7affd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4d7c15ca50 0x7f4d7c06b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53094/0 (socket says 172.21.15.144:53094) 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d7affd700 1 -- 172.21.15.144:0/1173355946 learned_addr learned my addr 172.21.15.144:0/1173355946 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d7b7fe700 1 --2- 172.21.15.144:0/1173355946 >> v2:172.21.15.177:3300/0 conn(0x7f4d7c0ccc40 0x7f4d7c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d7a7fc700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d7c0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354499711 0 0) 0x7f4d7c0cf5e0 con 0x7f4d7c15ca50 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d640035b0 con 0x7f4d7c15ca50 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2860662092 0 0) 0x7f4d640035b0 con 0x7f4d7c15ca50 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 msgr2=0x7f4d7c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d7c0cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 >> v2:172.21.15.177:3300/0 conn(0x7f4d7c0ccc40 msgr2=0x7f4d7c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 --2- 172.21.15.144:0/1173355946 >> v2:172.21.15.177:3300/0 conn(0x7f4d7c0ccc40 0x7f4d7c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0cef90 con 0x7f4d7c15ca50 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f4d70001670 con 0x7f4d7c15ca50 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d7a7fc700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d7c0cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 >> v1:172.21.15.144:6789/0 conn(0x7f4d7c15ca50 legacy=0x7f4d7c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d64003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d64004d90 0x7f4d64005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cd470 con 0x7f4d64003a00 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cecb0 con 0x7f4d64004d90 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6400ba10 con 0x7f4d7c06ba90 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.471+0000 7f4d7b7fe700 1 --2- 172.21.15.144:0/1173355946 >> v2:172.21.15.177:3300/0 conn(0x7f4d7c0ccc40 0x7f4d7c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:12.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d64003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7a7fc700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7b7fe700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d64004d90 0x7f4d64005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 -- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 msgr2=0x7f4d64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c06ba90 0x7f4d64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 -- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d64004d90 msgr2=0x7f4d64005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d64004d90 0x7f4d64005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 -- 172.21.15.144:0/1173355946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0094a0 con 0x7f4d64003a00 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d7affd700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d64003e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c320 tx=0x7f4d64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6588c507565d396a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000ce00 con 0x7f4d64003a00 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d70005960 con 0x7f4d64003a00 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.472+0000 7f4d79ffb700 1 -- 172.21.15.144:0/1173355946 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70005d10 con 0x7f4d64003a00 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d812e0700 1 -- 172.21.15.144:0/1173355946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 msgr2=0x7f4d64003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d812e0700 1 --2- 172.21.15.144:0/1173355946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d64003e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c320 tx=0x7f4d64005880 comp rx=0 tx=0).stop 2023-11-28T11:52:12.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d7b7fe700 1 -- 172.21.15.144:0/1173355946 reap_dead start 2023-11-28T11:52:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d812e0700 1 -- 172.21.15.144:0/1173355946 shutdown_connections 2023-11-28T11:52:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d812e0700 1 -- 172.21.15.144:0/1173355946 >> 172.21.15.144:0/1173355946 conn(0x7f4d7c09a230 msgr2=0x7f4d7c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d812e0700 1 -- 172.21.15.144:0/1173355946 shutdown_connections 2023-11-28T11:52:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.473+0000 7f4d812e0700 1 -- 172.21.15.144:0/1173355946 wait complete. 2023-11-28T11:52:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.474+0000 7f4d812e0700 1 Processor -- start 2023-11-28T11:52:12.478 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.474+0000 7f4d812e0700 1 -- start start 2023-11-28T11:52:12.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d7c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c15d190 0x7f4d7c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c1572a0 0x7f4d7c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c1600a0 con 0x7f4d64003a00 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15d600 con 0x7f4d7c15d190 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15d890 con 0x7f4d7c1572a0 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d7c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d7c0c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36394/0 (socket says 172.21.15.144:36394) 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 -- 172.21.15.144:0/2395684163 learned_addr learned my addr 172.21.15.144:0/2395684163 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 -- 172.21.15.144:0/2395684163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c1572a0 msgr2=0x7f4d7c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c1572a0 0x7f4d7c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 -- 172.21.15.144:0/2395684163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c15d190 msgr2=0x7f4d7c15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c15d190 0x7f4d7c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 -- 172.21.15.144:0/2395684163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0cd0e0 con 0x7f4d64003a00 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d7affd700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d7c0c49c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d70008fd0 tx=0x7f4d70001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:12.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70000ed0 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d70004a80 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c0cf270 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004c30 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.475+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c15ce20 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.476+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c0745c0 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.477+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d70024030 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.477+0000 7f4d637fe700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4d680682f0 0x7f4d6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.477+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d700870b0 con 0x7f4d64003a00 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.477+0000 7f4d7a7fc700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4d680682f0 0x7f4d6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.478+0000 7f4d7a7fc700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4d680682f0 0x7f4d6806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06c750 tx=0x7f4d6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:12.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.482+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d700539d0 con 0x7f4d64003a00 2023-11-28T11:52:12.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.706+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d7c1e6460 con 0x7f4d64003a00 2023-11-28T11:52:12.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.707+0000 7f4d637fe700 1 -- 172.21.15.144:0/2395684163 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f4d70007da0 con 0x7f4d64003a00 2023-11-28T11:52:12.708 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:12.708 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4d680682f0 msgr2=0x7f4d6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4d680682f0 0x7f4d6806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06c750 tx=0x7f4d6c008040 comp rx=0 tx=0).stop 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 msgr2=0x7f4d7c0c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d7c0c49c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d70008fd0 tx=0x7f4d70001a00 comp rx=0 tx=0).stop 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 shutdown_connections 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d64003a00 0x7f4d7c0c49c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4d680682f0 0x7f4d6806a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d7c1572a0 0x7f4d7c0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 --2- 172.21.15.144:0/2395684163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d7c15d190 0x7f4d7c15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 >> 172.21.15.144:0/2395684163 conn(0x7f4d7c09a230 msgr2=0x7f4d7c15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:12.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 shutdown_connections 2023-11-28T11:52:12.711 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.709+0000 7f4d812e0700 1 -- 172.21.15.144:0/2395684163 wait complete. 2023-11-28T11:52:12.711 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:12.722 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:12.806 INFO:tasks.ceph.mds.a.smithi144.stderr:2023-11-28T11:52:12.806+0000 7ff8c3457700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:52:12.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.841+0000 7fac107a4700 1 Processor -- start 2023-11-28T11:52:12.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.842+0000 7fac107a4700 1 -- start start 2023-11-28T11:52:12.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.842+0000 7fac107a4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fac080cd8c0 0x7fac08140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac107a4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0813ff00 con 0x7fac080cdc90 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac107a4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac081404c0 con 0x7fac0806f850 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac107a4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0815aa80 con 0x7fac080cd8c0 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0dd3f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fac080cdc90 0x7fac080b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53114/0 (socket says 172.21.15.144:53114) 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0dd3f700 1 -- 172.21.15.144:0/612434693 learned_addr learned my addr 172.21.15.144:0/612434693 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0e540700 1 --2- 172.21.15.144:0/612434693 >> v2:172.21.15.177:3300/0 conn(0x7fac080cd8c0 0x7fac08140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909357955 0 0) 0x7fac0813ff00 con 0x7fac080cdc90 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf40035b0 con 0x7fac080cdc90 2023-11-28T11:52:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291775874 0 0) 0x7fabf40035b0 con 0x7fac080cdc90 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 >> v2:172.21.15.177:3300/0 conn(0x7fac080cd8c0 msgr2=0x7fac08140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 --2- 172.21.15.144:0/612434693 >> v2:172.21.15.177:3300/0 conn(0x7fac080cd8c0 0x7fac08140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 >> v1:172.21.15.177:6789/0 conn(0x7fac0806f850 legacy=0x7fac0806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080b3040 con 0x7fac080cdc90 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fac04001670 con 0x7fac080cdc90 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 >> v1:172.21.15.144:6789/0 conn(0x7fac080cdc90 legacy=0x7fac080b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf4003a00 0x7fabf4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fabf4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf4005200 0x7fabf400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0815aa80 con 0x7fabf4004850 2023-11-28T11:52:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf400c110 con 0x7fabf4005200 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf400c2d0 con 0x7fabf4003a00 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0dd3f700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fabf4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0e540700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf4003a00 0x7fabf4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.843+0000 7fac0ed41700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf4005200 0x7fabf400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0dd3f700 1 -- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf4003a00 msgr2=0x7fabf4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0dd3f700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf4003a00 0x7fabf4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0dd3f700 1 -- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf4005200 msgr2=0x7fabf400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0dd3f700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf4005200 0x7fabf400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0dd3f700 1 -- 172.21.15.144:0/612434693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf40035b0 con 0x7fabf4004850 2023-11-28T11:52:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0ed41700 1 -- 172.21.15.144:0/612434693 reap_dead start 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0dd3f700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fabf4004cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac04001050 tx=0x7fabf4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7823e3ca97f23b83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0400cd50 con 0x7fabf4004850 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac04005960 con 0x7fabf4004850 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac0d53e700 1 -- 172.21.15.144:0/612434693 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04005d10 con 0x7fabf4004850 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac107a4700 1 -- 172.21.15.144:0/612434693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 msgr2=0x7fabf4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.844+0000 7fac107a4700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fabf4004cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac04001050 tx=0x7fabf4005880 comp rx=0 tx=0).stop 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.845+0000 7fac107a4700 1 -- 172.21.15.144:0/612434693 shutdown_connections 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.845+0000 7fac107a4700 1 --2- 172.21.15.144:0/612434693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fabf4004cc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.845+0000 7fac107a4700 1 -- 172.21.15.144:0/612434693 >> 172.21.15.144:0/612434693 conn(0x7fac0806aed0 msgr2=0x7fac0806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.845+0000 7fac107a4700 1 -- 172.21.15.144:0/612434693 shutdown_connections 2023-11-28T11:52:12.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.845+0000 7fac107a4700 1 -- 172.21.15.144:0/612434693 wait complete. 2023-11-28T11:52:12.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.846+0000 7fac107a4700 1 Processor -- start 2023-11-28T11:52:12.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.846+0000 7fac107a4700 1 -- start start 2023-11-28T11:52:12.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fac081f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac08140ff0 0x7fac08142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac08142520 0x7fac080c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac08141480 con 0x7fabf4004850 2023-11-28T11:52:12.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac08141760 con 0x7fac08142520 2023-11-28T11:52:12.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac08141a40 con 0x7fac08140ff0 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fac081f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fac081f40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36414/0 (socket says 172.21.15.144:36414) 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 -- 172.21.15.144:0/1561721753 learned_addr learned my addr 172.21.15.144:0/1561721753 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 -- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac08140ff0 msgr2=0x7fac08142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac08140ff0 0x7fac08142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 -- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac08142520 msgr2=0x7fac080c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0ed41700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac08142520 0x7fac080c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac08142520 0x7fac080c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 -- 172.21.15.144:0/1561721753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080b35f0 con 0x7fabf4004850 2023-11-28T11:52:12.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac0e540700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fac081f40b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fabf8011920 tx=0x7fabf8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8009610 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf8005c30 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac080b3310 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800f730 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.847+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac08140150 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.848+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac08068690 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.849+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fabf800f890 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.849+0000 7fabfeffd700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fabec0682f0 0x7fabec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.849+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabf8086530 con 0x7fabf4004850 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.849+0000 7fac0dd3f700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fabec0682f0 0x7fabec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:12.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.850+0000 7fac0dd3f700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fabec0682f0 0x7fabec06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac08144280 tx=0x7fac0400f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:12.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:12.853+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fabf8052e40 con 0x7fabf4004850 2023-11-28T11:52:13.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.036+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac0813e930 con 0x7fabf4004850 2023-11-28T11:52:13.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.037+0000 7fabfeffd700 1 -- 172.21.15.144:0/1561721753 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fabf8056460 con 0x7fabf4004850 2023-11-28T11:52:13.038 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:13.038 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:10.310687+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fabec0682f0 msgr2=0x7fabec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fabec0682f0 0x7fabec06a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac08144280 tx=0x7fac0400f000 comp rx=0 tx=0).stop 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 msgr2=0x7fac081f40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fac081f40b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fabf8011920 tx=0x7fabf8011ce0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 shutdown_connections 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf4004850 0x7fac081f40b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fabec0682f0 0x7fabec06a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac08142520 0x7fac080c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 --2- 172.21.15.144:0/1561721753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac08140ff0 0x7fac08142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 >> 172.21.15.144:0/1561721753 conn(0x7fac0806aed0 msgr2=0x7fac0806fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:13.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 shutdown_connections 2023-11-28T11:52:13.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.039+0000 7fac107a4700 1 -- 172.21.15.144:0/1561721753 wait complete. 2023-11-28T11:52:13.052 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:52:13.052 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.171+0000 7f2a9e9e3700 1 Processor -- start 2023-11-28T11:52:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.171+0000 7f2a9e9e3700 1 -- start start 2023-11-28T11:52:13.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a9e9e3700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2a980ce040 0x7f2a9815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a9e9e3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2a980b2930 0x7f2a980b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a9e9e3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9806b420 con 0x7f2a980d0a00 2023-11-28T11:52:13.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a9e9e3700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980b32f0 con 0x7f2a980ce040 2023-11-28T11:52:13.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a9e9e3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9806ff60 con 0x7f2a980b2930 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a97fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2a980ce040 0x7f2a9815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a977fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f2a980d0a00 0x7f2a9806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53134/0 (socket says 172.21.15.144:53134) 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a977fe700 1 -- 172.21.15.144:0/4288358962 learned_addr learned my addr 172.21.15.144:0/4288358962 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a96ffd700 1 -- 172.21.15.144:0/4288358962 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186133808 0 0) 0x7f2a9806b420 con 0x7f2a980d0a00 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a96ffd700 1 -- 172.21.15.144:0/4288358962 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a800035b0 con 0x7f2a980d0a00 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a9cf80700 1 --2- 172.21.15.144:0/4288358962 >> v2:172.21.15.177:3300/0 conn(0x7f2a980b2930 0x7f2a980b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a97fff700 1 -- 172.21.15.144:0/4288358962 >> v2:172.21.15.177:3300/0 conn(0x7f2a980b2930 msgr2=0x7f2a980b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a97fff700 1 --2- 172.21.15.144:0/4288358962 >> v2:172.21.15.177:3300/0 conn(0x7f2a980b2930 0x7f2a980b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a97fff700 1 -- 172.21.15.144:0/4288358962 >> v1:172.21.15.144:6789/0 conn(0x7f2a980d0a00 legacy=0x7f2a9806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a97fff700 1 -- 172.21.15.144:0/4288358962 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9806b700 con 0x7f2a980ce040 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a97fff700 1 --2- 172.21.15.144:0/4288358962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980ce040 0x7f2a9815df10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2a98073df0 tx=0x7f2a8c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64fdbf510a7c29c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.172+0000 7f2a96ffd700 1 -- 172.21.15.144:0/4288358962 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c005e30 con 0x7f2a980ce040 2023-11-28T11:52:13.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a96ffd700 1 -- 172.21.15.144:0/4288358962 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c006030 con 0x7f2a980ce040 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a96ffd700 1 -- 172.21.15.144:0/4288358962 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0063c0 con 0x7f2a980ce040 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/4288358962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980ce040 msgr2=0x7f2a9815df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/4288358962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980ce040 0x7f2a9815df10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2a98073df0 tx=0x7f2a8c009b00 comp rx=0 tx=0).stop 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/4288358962 shutdown_connections 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/4288358962 >> v2:172.21.15.177:3300/0 conn(0x7f2a980b2930 0x7f2a980b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/4288358962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980ce040 0x7f2a9815df10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/4288358962 >> 172.21.15.144:0/4288358962 conn(0x7f2a9806aed0 msgr2=0x7f2a9806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/4288358962 shutdown_connections 2023-11-28T11:52:13.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.173+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/4288358962 wait complete. 2023-11-28T11:52:13.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.174+0000 7f2a9e9e3700 1 Processor -- start 2023-11-28T11:52:13.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.174+0000 7f2a9e9e3700 1 -- start start 2023-11-28T11:52:13.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9e9e3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a980b2930 0x7f2a9814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9e9e3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a980ce040 0x7f2a980b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9e9e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 0x7f2a9815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9e9e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980c57e0 con 0x7f2a980d0a00 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9e9e3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9814a8b0 con 0x7f2a980b2930 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9e9e3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980c4b60 con 0x7f2a980ce040 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9cf80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 0x7f2a9815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9cf80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 0x7f2a9815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36432/0 (socket says 172.21.15.144:36432) 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a9cf80700 1 -- 172.21.15.144:0/1486449026 learned_addr learned my addr 172.21.15.144:0/1486449026 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.175+0000 7f2a97fff700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a980b2930 0x7f2a9814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a977fe700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a980ce040 0x7f2a980b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9cf80700 1 -- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a980ce040 msgr2=0x7f2a980b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9cf80700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a980ce040 0x7f2a980b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9cf80700 1 -- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a980b2930 msgr2=0x7f2a9814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9cf80700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a980b2930 0x7f2a9814a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9cf80700 1 -- 172.21.15.144:0/1486449026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c009430 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9cf80700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 0x7f2a9815a7f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000cc30 tx=0x7f2a9000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000ab70 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9000acd0 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9806bcc0 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90003c40 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.176+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9806b9e0 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.177+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9806bfa0 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.177+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a9001f070 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.178+0000 7f2a94ff9700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2a840682f0 0x7f2a8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.178+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a90085d70 con 0x7f2a980d0a00 2023-11-28T11:52:13.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.178+0000 7f2a97fff700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2a840682f0 0x7f2a8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.178+0000 7f2a97fff700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2a840682f0 0x7f2a8406a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000c00 tx=0x7f2a8c008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.182+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a900525d0 con 0x7f2a980d0a00 2023-11-28T11:52:13.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.360+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a98073ad0 con 0x7f2a840682f0 2023-11-28T11:52:13.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.368+0000 7f2a94ff9700 1 -- 172.21.15.144:0/1486449026 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290061 (secure 0 0 0) 0x7f2a98073ad0 con 0x7f2a840682f0 2023-11-28T11:52:13.372 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2a840682f0 msgr2=0x7f2a8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2a840682f0 0x7f2a8406a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000c00 tx=0x7f2a8c008070 comp rx=0 tx=0).stop 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 msgr2=0x7f2a9815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 0x7f2a9815a7f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000cc30 tx=0x7f2a9000a1f0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 shutdown_connections 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a980d0a00 0x7f2a9815a7f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2a840682f0 0x7f2a8406a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a980b2930 0x7f2a9814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 --2- 172.21.15.144:0/1486449026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a980ce040 0x7f2a980b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 >> 172.21.15.144:0/1486449026 conn(0x7f2a9806aed0 msgr2=0x7f2a9806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:13.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 shutdown_connections 2023-11-28T11:52:13.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.372+0000 7f2a9e9e3700 1 -- 172.21.15.144:0/1486449026 wait complete. 2023-11-28T11:52:13.376 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:13.386 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-11-28T11:52:12.494421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":68844,"kb_used_data":4012,"kb_used_omap":0,"kb_used_meta":64768,"kb_avail":754905876,"statfs":{"total":773094113280,"available":773023617024,"internally_reserved":0,"allocated":4108288,"data_stored":2337334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66322432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.590254"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332498+0000","last_change":"2023-11-28T11:52:09.329046+0000","last_active":"2023-11-28T11:52:09.332498+0000","last_peered":"2023-11-28T11:52:09.332498+0000","last_clean":"2023-11-28T11:52:09.332498+0000","last_became_active":"2023-11-28T11:52:09.328834+0000","last_became_peered":"2023-11-28T11:52:09.328834+0000","last_unstale":"2023-11-28T11:52:09.332498+0000","last_undegraded":"2023-11-28T11:52:09.332498+0000","last_fullsized":"2023-11-28T11:52:09.332498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:47.116622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331041+0000","last_change":"2023-11-28T11:52:09.324674+0000","last_active":"2023-11-28T11:52:09.331041+0000","last_peered":"2023-11-28T11:52:09.331041+0000","last_clean":"2023-11-28T11:52:09.331041+0000","last_became_active":"2023-11-28T11:52:09.324411+0000","last_became_peered":"2023-11-28T11:52:09.324411+0000","last_unstale":"2023-11-28T11:52:09.331041+0000","last_undegraded":"2023-11-28T11:52:09.331041+0000","last_fullsized":"2023-11-28T11:52:09.331041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:11.323168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336171+0000","last_change":"2023-11-28T11:52:09.335158+0000","last_active":"2023-11-28T11:52:09.336171+0000","last_peered":"2023-11-28T11:52:09.336171+0000","last_clean":"2023-11-28T11:52:09.336171+0000","last_became_active":"2023-11-28T11:52:09.334912+0000","last_became_peered":"2023-11-28T11:52:09.334912+0000","last_unstale":"2023-11-28T11:52:09.336171+0000","last_undegraded":"2023-11-28T11:52:09.336171+0000","last_fullsized":"2023-11-28T11:52:09.336171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:07.358461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336021+0000","last_change":"2023-11-28T11:52:09.334226+0000","last_active":"2023-11-28T11:52:09.336021+0000","last_peered":"2023-11-28T11:52:09.336021+0000","last_clean":"2023-11-28T11:52:09.336021+0000","last_became_active":"2023-11-28T11:52:09.333892+0000","last_became_peered":"2023-11-28T11:52:09.333892+0000","last_unstale":"2023-11-28T11:52:09.336021+0000","last_undegraded":"2023-11-28T11:52:09.336021+0000","last_fullsized":"2023-11-28T11:52:09.336021+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:56.578314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.335919+0000","last_change":"2023-11-28T11:52:09.333770+0000","last_active":"2023-11-28T11:52:09.335919+0000","last_peered":"2023-11-28T11:52:09.335919+0000","last_clean":"2023-11-28T11:52:09.335919+0000","last_became_active":"2023-11-28T11:52:09.333502+0000","last_became_peered":"2023-11-28T11:52:09.333502+0000","last_unstale":"2023-11-28T11:52:09.335919+0000","last_undegraded":"2023-11-28T11:52:09.335919+0000","last_fullsized":"2023-11-28T11:52:09.335919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:40.792764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341345+0000","last_change":"2023-11-28T11:52:09.330238+0000","last_active":"2023-11-28T11:52:09.341345+0000","last_peered":"2023-11-28T11:52:09.341345+0000","last_clean":"2023-11-28T11:52:09.341345+0000","last_became_active":"2023-11-28T11:52:09.330022+0000","last_became_peered":"2023-11-28T11:52:09.330022+0000","last_unstale":"2023-11-28T11:52:09.341345+0000","last_undegraded":"2023-11-28T11:52:09.341345+0000","last_fullsized":"2023-11-28T11:52:09.341345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:29.214687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336803+0000","last_change":"2023-11-28T11:52:09.332627+0000","last_active":"2023-11-28T11:52:09.336803+0000","last_peered":"2023-11-28T11:52:09.336803+0000","last_clean":"2023-11-28T11:52:09.336803+0000","last_became_active":"2023-11-28T11:52:09.332334+0000","last_became_peered":"2023-11-28T11:52:09.332334+0000","last_unstale":"2023-11-28T11:52:09.336803+0000","last_undegraded":"2023-11-28T11:52:09.336803+0000","last_fullsized":"2023-11-28T11:52:09.336803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:11.323168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.330949+0000","last_change":"2023-11-28T11:52:09.323546+0000","last_active":"2023-11-28T11:52:09.330949+0000","last_peered":"2023-11-28T11:52:09.330949+0000","last_clean":"2023-11-28T11:52:09.330949+0000","last_became_active":"2023-11-28T11:52:09.323304+0000","last_became_peered":"2023-11-28T11:52:09.323304+0000","last_unstale":"2023-11-28T11:52:09.330949+0000","last_undegraded":"2023-11-28T11:52:09.330949+0000","last_fullsized":"2023-11-28T11:52:09.330949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:15:30.861236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336175+0000","last_change":"2023-11-28T11:52:09.335162+0000","last_active":"2023-11-28T11:52:09.336175+0000","last_peered":"2023-11-28T11:52:09.336175+0000","last_clean":"2023-11-28T11:52:09.336175+0000","last_became_active":"2023-11-28T11:52:09.334875+0000","last_became_peered":"2023-11-28T11:52:09.334875+0000","last_unstale":"2023-11-28T11:52:09.336175+0000","last_undegraded":"2023-11-28T11:52:09.336175+0000","last_fullsized":"2023-11-28T11:52:09.336175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:20:59.977365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336859+0000","last_change":"2023-11-28T11:52:09.330989+0000","last_active":"2023-11-28T11:52:09.336859+0000","last_peered":"2023-11-28T11:52:09.336859+0000","last_clean":"2023-11-28T11:52:09.336859+0000","last_became_active":"2023-11-28T11:52:09.330760+0000","last_became_peered":"2023-11-28T11:52:09.330760+0000","last_unstale":"2023-11-28T11:52:09.336859+0000","last_undegraded":"2023-11-28T11:52:09.336859+0000","last_fullsized":"2023-11-28T11:52:09.336859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:28.093634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336918+0000","last_change":"2023-11-28T11:52:09.333179+0000","last_active":"2023-11-28T11:52:09.336918+0000","last_peered":"2023-11-28T11:52:09.336918+0000","last_clean":"2023-11-28T11:52:09.336918+0000","last_became_active":"2023-11-28T11:52:09.332920+0000","last_became_peered":"2023-11-28T11:52:09.332920+0000","last_unstale":"2023-11-28T11:52:09.336918+0000","last_undegraded":"2023-11-28T11:52:09.336918+0000","last_fullsized":"2023-11-28T11:52:09.336918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:24.177480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325926+0000","last_change":"2023-11-28T11:52:09.323600+0000","last_active":"2023-11-28T11:52:09.325926+0000","last_peered":"2023-11-28T11:52:09.325926+0000","last_clean":"2023-11-28T11:52:09.325926+0000","last_became_active":"2023-11-28T11:52:09.323332+0000","last_became_peered":"2023-11-28T11:52:09.323332+0000","last_unstale":"2023-11-28T11:52:09.325926+0000","last_undegraded":"2023-11-28T11:52:09.325926+0000","last_fullsized":"2023-11-28T11:52:09.325926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:04:37.612996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.337105+0000","last_change":"2023-11-28T11:52:09.330628+0000","last_active":"2023-11-28T11:52:09.337105+0000","last_peered":"2023-11-28T11:52:09.337105+0000","last_clean":"2023-11-28T11:52:09.337105+0000","last_became_active":"2023-11-28T11:52:09.330332+0000","last_became_peered":"2023-11-28T11:52:09.330332+0000","last_unstale":"2023-11-28T11:52:09.337105+0000","last_undegraded":"2023-11-28T11:52:09.337105+0000","last_fullsized":"2023-11-28T11:52:09.337105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:39.431994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340190+0000","last_change":"2023-11-28T11:52:09.334264+0000","last_active":"2023-11-28T11:52:09.340190+0000","last_peered":"2023-11-28T11:52:09.340190+0000","last_clean":"2023-11-28T11:52:09.340190+0000","last_became_active":"2023-11-28T11:52:09.333315+0000","last_became_peered":"2023-11-28T11:52:09.333315+0000","last_unstale":"2023-11-28T11:52:09.340190+0000","last_undegraded":"2023-11-28T11:52:09.340190+0000","last_fullsized":"2023-11-28T11:52:09.340190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:28.093634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.326603+0000","last_change":"2023-11-28T11:52:09.320288+0000","last_active":"2023-11-28T11:52:09.326603+0000","last_peered":"2023-11-28T11:52:09.326603+0000","last_clean":"2023-11-28T11:52:09.326603+0000","last_became_active":"2023-11-28T11:52:09.319977+0000","last_became_peered":"2023-11-28T11:52:09.319977+0000","last_unstale":"2023-11-28T11:52:09.326603+0000","last_undegraded":"2023-11-28T11:52:09.326603+0000","last_fullsized":"2023-11-28T11:52:09.326603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:07:09.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336483+0000","last_change":"2023-11-28T11:52:09.327570+0000","last_active":"2023-11-28T11:52:09.336483+0000","last_peered":"2023-11-28T11:52:09.336483+0000","last_clean":"2023-11-28T11:52:09.336483+0000","last_became_active":"2023-11-28T11:52:09.327037+0000","last_became_peered":"2023-11-28T11:52:09.327037+0000","last_unstale":"2023-11-28T11:52:09.336483+0000","last_undegraded":"2023-11-28T11:52:09.336483+0000","last_fullsized":"2023-11-28T11:52:09.336483+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:47.116622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336660+0000","last_change":"2023-11-28T11:52:09.327560+0000","last_active":"2023-11-28T11:52:09.336660+0000","last_peered":"2023-11-28T11:52:09.336660+0000","last_clean":"2023-11-28T11:52:09.336660+0000","last_became_active":"2023-11-28T11:52:09.327251+0000","last_became_peered":"2023-11-28T11:52:09.327251+0000","last_unstale":"2023-11-28T11:52:09.336660+0000","last_undegraded":"2023-11-28T11:52:09.336660+0000","last_fullsized":"2023-11-28T11:52:09.336660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:39.431994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.344792+0000","last_change":"2023-11-28T11:52:09.340087+0000","last_active":"2023-11-28T11:52:09.344792+0000","last_peered":"2023-11-28T11:52:09.344792+0000","last_clean":"2023-11-28T11:52:09.344792+0000","last_became_active":"2023-11-28T11:52:09.339836+0000","last_became_peered":"2023-11-28T11:52:09.339836+0000","last_unstale":"2023-11-28T11:52:09.344792+0000","last_undegraded":"2023-11-28T11:52:09.344792+0000","last_fullsized":"2023-11-28T11:52:09.344792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.578877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.329491+0000","last_change":"2023-11-28T11:52:09.327409+0000","last_active":"2023-11-28T11:52:09.329491+0000","last_peered":"2023-11-28T11:52:09.329491+0000","last_clean":"2023-11-28T11:52:09.329491+0000","last_became_active":"2023-11-28T11:52:09.327137+0000","last_became_peered":"2023-11-28T11:52:09.327137+0000","last_unstale":"2023-11-28T11:52:09.329491+0000","last_undegraded":"2023-11-28T11:52:09.329491+0000","last_fullsized":"2023-11-28T11:52:09.329491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:00.508719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342107+0000","last_change":"2023-11-28T11:52:09.338731+0000","last_active":"2023-11-28T11:52:09.342107+0000","last_peered":"2023-11-28T11:52:09.342107+0000","last_clean":"2023-11-28T11:52:09.342107+0000","last_became_active":"2023-11-28T11:52:09.338412+0000","last_became_peered":"2023-11-28T11:52:09.338412+0000","last_unstale":"2023-11-28T11:52:09.342107+0000","last_undegraded":"2023-11-28T11:52:09.342107+0000","last_fullsized":"2023-11-28T11:52:09.342107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:46.376063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.334499+0000","last_change":"2023-11-28T11:52:09.325697+0000","last_active":"2023-11-28T11:52:09.334499+0000","last_peered":"2023-11-28T11:52:09.334499+0000","last_clean":"2023-11-28T11:52:09.334499+0000","last_became_active":"2023-11-28T11:52:09.325415+0000","last_became_peered":"2023-11-28T11:52:09.325415+0000","last_unstale":"2023-11-28T11:52:09.334499+0000","last_undegraded":"2023-11-28T11:52:09.334499+0000","last_fullsized":"2023-11-28T11:52:09.334499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:17.206689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340213+0000","last_change":"2023-11-28T11:52:09.336362+0000","last_active":"2023-11-28T11:52:09.340213+0000","last_peered":"2023-11-28T11:52:09.340213+0000","last_clean":"2023-11-28T11:52:09.340213+0000","last_became_active":"2023-11-28T11:52:09.336071+0000","last_became_peered":"2023-11-28T11:52:09.336071+0000","last_unstale":"2023-11-28T11:52:09.340213+0000","last_undegraded":"2023-11-28T11:52:09.340213+0000","last_fullsized":"2023-11-28T11:52:09.340213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:37.353038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.330225+0000","last_change":"2023-11-28T11:52:09.324610+0000","last_active":"2023-11-28T11:52:09.330225+0000","last_peered":"2023-11-28T11:52:09.330225+0000","last_clean":"2023-11-28T11:52:09.330225+0000","last_became_active":"2023-11-28T11:52:09.323323+0000","last_became_peered":"2023-11-28T11:52:09.323323+0000","last_unstale":"2023-11-28T11:52:09.330225+0000","last_undegraded":"2023-11-28T11:52:09.330225+0000","last_fullsized":"2023-11-28T11:52:09.330225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:24.177480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325971+0000","last_change":"2023-11-28T11:52:09.324187+0000","last_active":"2023-11-28T11:52:09.325971+0000","last_peered":"2023-11-28T11:52:09.325971+0000","last_clean":"2023-11-28T11:52:09.325971+0000","last_became_active":"2023-11-28T11:52:09.323880+0000","last_became_peered":"2023-11-28T11:52:09.323880+0000","last_unstale":"2023-11-28T11:52:09.325971+0000","last_undegraded":"2023-11-28T11:52:09.325971+0000","last_fullsized":"2023-11-28T11:52:09.325971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:08.609403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331768+0000","last_change":"2023-11-28T11:52:09.327794+0000","last_active":"2023-11-28T11:52:09.331768+0000","last_peered":"2023-11-28T11:52:09.331768+0000","last_clean":"2023-11-28T11:52:09.331768+0000","last_became_active":"2023-11-28T11:52:09.327512+0000","last_became_peered":"2023-11-28T11:52:09.327512+0000","last_unstale":"2023-11-28T11:52:09.331768+0000","last_undegraded":"2023-11-28T11:52:09.331768+0000","last_fullsized":"2023-11-28T11:52:09.331768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:15.571841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341412+0000","last_change":"2023-11-28T11:52:09.335760+0000","last_active":"2023-11-28T11:52:09.341412+0000","last_peered":"2023-11-28T11:52:09.341412+0000","last_clean":"2023-11-28T11:52:09.341412+0000","last_became_active":"2023-11-28T11:52:09.335479+0000","last_became_peered":"2023-11-28T11:52:09.335479+0000","last_unstale":"2023-11-28T11:52:09.341412+0000","last_undegraded":"2023-11-28T11:52:09.341412+0000","last_fullsized":"2023-11-28T11:52:09.341412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:31.609131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331774+0000","last_change":"2023-11-28T11:52:09.329173+0000","last_active":"2023-11-28T11:52:09.331774+0000","last_peered":"2023-11-28T11:52:09.331774+0000","last_clean":"2023-11-28T11:52:09.331774+0000","last_became_active":"2023-11-28T11:52:09.328857+0000","last_became_peered":"2023-11-28T11:52:09.328857+0000","last_unstale":"2023-11-28T11:52:09.331774+0000","last_undegraded":"2023-11-28T11:52:09.331774+0000","last_fullsized":"2023-11-28T11:52:09.331774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:10.950951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336776+0000","last_change":"2023-11-28T11:52:09.332719+0000","last_active":"2023-11-28T11:52:09.336776+0000","last_peered":"2023-11-28T11:52:09.336776+0000","last_clean":"2023-11-28T11:52:09.336776+0000","last_became_active":"2023-11-28T11:52:09.332434+0000","last_became_peered":"2023-11-28T11:52:09.332434+0000","last_unstale":"2023-11-28T11:52:09.336776+0000","last_undegraded":"2023-11-28T11:52:09.336776+0000","last_fullsized":"2023-11-28T11:52:09.336776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:09.018975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331159+0000","last_change":"2023-11-28T11:52:09.326773+0000","last_active":"2023-11-28T11:52:09.331159+0000","last_peered":"2023-11-28T11:52:09.331159+0000","last_clean":"2023-11-28T11:52:09.331159+0000","last_became_active":"2023-11-28T11:52:09.326452+0000","last_became_peered":"2023-11-28T11:52:09.326452+0000","last_unstale":"2023-11-28T11:52:09.331159+0000","last_undegraded":"2023-11-28T11:52:09.331159+0000","last_fullsized":"2023-11-28T11:52:09.331159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:29.214687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.333590+0000","last_change":"2023-11-28T11:52:09.331101+0000","last_active":"2023-11-28T11:52:09.333590+0000","last_peered":"2023-11-28T11:52:09.333590+0000","last_clean":"2023-11-28T11:52:09.333590+0000","last_became_active":"2023-11-28T11:52:09.330801+0000","last_became_peered":"2023-11-28T11:52:09.330801+0000","last_unstale":"2023-11-28T11:52:09.333590+0000","last_undegraded":"2023-11-28T11:52:09.333590+0000","last_fullsized":"2023-11-28T11:52:09.333590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:26.954458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.330246+0000","last_change":"2023-11-28T11:52:09.324800+0000","last_active":"2023-11-28T11:52:09.330246+0000","last_peered":"2023-11-28T11:52:09.330246+0000","last_clean":"2023-11-28T11:52:09.330246+0000","last_became_active":"2023-11-28T11:52:09.323685+0000","last_became_peered":"2023-11-28T11:52:09.323685+0000","last_unstale":"2023-11-28T11:52:09.330246+0000","last_undegraded":"2023-11-28T11:52:09.330246+0000","last_fullsized":"2023-11-28T11:52:09.330246+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:48.462626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.324238+0000","last_change":"2023-11-28T11:52:09.322618+0000","last_active":"2023-11-28T11:52:09.324238+0000","last_peered":"2023-11-28T11:52:09.324238+0000","last_clean":"2023-11-28T11:52:09.324238+0000","last_became_active":"2023-11-28T11:52:09.322358+0000","last_became_peered":"2023-11-28T11:52:09.322358+0000","last_unstale":"2023-11-28T11:52:09.324238+0000","last_undegraded":"2023-11-28T11:52:09.324238+0000","last_fullsized":"2023-11-28T11:52:09.324238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:01.884916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336695+0000","last_change":"2023-11-28T11:52:09.328737+0000","last_active":"2023-11-28T11:52:09.336695+0000","last_peered":"2023-11-28T11:52:09.336695+0000","last_clean":"2023-11-28T11:52:09.336695+0000","last_became_active":"2023-11-28T11:52:09.328289+0000","last_became_peered":"2023-11-28T11:52:09.328289+0000","last_unstale":"2023-11-28T11:52:09.336695+0000","last_undegraded":"2023-11-28T11:52:09.336695+0000","last_fullsized":"2023-11-28T11:52:09.336695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:29:02.011479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342529+0000","last_change":"2023-11-28T11:52:08.922237+0000","last_active":"2023-11-28T11:52:09.342529+0000","last_peered":"2023-11-28T11:52:09.342529+0000","last_clean":"2023-11-28T11:52:09.342529+0000","last_became_active":"2023-11-28T11:52:08.921926+0000","last_became_peered":"2023-11-28T11:52:08.921926+0000","last_unstale":"2023-11-28T11:52:09.342529+0000","last_undegraded":"2023-11-28T11:52:09.342529+0000","last_fullsized":"2023-11-28T11:52:09.342529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:23:54.348920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341296+0000","last_change":"2023-11-28T11:52:09.329043+0000","last_active":"2023-11-28T11:52:09.341296+0000","last_peered":"2023-11-28T11:52:09.341296+0000","last_clean":"2023-11-28T11:52:09.341296+0000","last_became_active":"2023-11-28T11:52:09.328613+0000","last_became_peered":"2023-11-28T11:52:09.328613+0000","last_unstale":"2023-11-28T11:52:09.341296+0000","last_undegraded":"2023-11-28T11:52:09.341296+0000","last_fullsized":"2023-11-28T11:52:09.341296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:23:47.243899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336885+0000","last_change":"2023-11-28T11:52:08.920434+0000","last_active":"2023-11-28T11:52:09.336885+0000","last_peered":"2023-11-28T11:52:09.336885+0000","last_clean":"2023-11-28T11:52:09.336885+0000","last_became_active":"2023-11-28T11:52:08.920162+0000","last_became_peered":"2023-11-28T11:52:08.920162+0000","last_unstale":"2023-11-28T11:52:09.336885+0000","last_undegraded":"2023-11-28T11:52:09.336885+0000","last_fullsized":"2023-11-28T11:52:09.336885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:36.400378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339981+0000","last_change":"2023-11-28T11:52:09.334508+0000","last_active":"2023-11-28T11:52:09.339981+0000","last_peered":"2023-11-28T11:52:09.339981+0000","last_clean":"2023-11-28T11:52:09.339981+0000","last_became_active":"2023-11-28T11:52:09.334181+0000","last_became_peered":"2023-11-28T11:52:09.334181+0000","last_unstale":"2023-11-28T11:52:09.339981+0000","last_undegraded":"2023-11-28T11:52:09.339981+0000","last_fullsized":"2023-11-28T11:52:09.339981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:37.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339743+0000","last_change":"2023-11-28T11:52:08.928738+0000","last_active":"2023-11-28T11:52:09.339743+0000","last_peered":"2023-11-28T11:52:09.339743+0000","last_clean":"2023-11-28T11:52:09.339743+0000","last_became_active":"2023-11-28T11:52:08.928428+0000","last_became_peered":"2023-11-28T11:52:08.928428+0000","last_unstale":"2023-11-28T11:52:09.339743+0000","last_undegraded":"2023-11-28T11:52:09.339743+0000","last_fullsized":"2023-11-28T11:52:09.339743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:55.574517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.333207+0000","last_change":"2023-11-28T11:52:09.332189+0000","last_active":"2023-11-28T11:52:09.333207+0000","last_peered":"2023-11-28T11:52:09.333207+0000","last_clean":"2023-11-28T11:52:09.333207+0000","last_became_active":"2023-11-28T11:52:09.331957+0000","last_became_peered":"2023-11-28T11:52:09.331957+0000","last_unstale":"2023-11-28T11:52:09.333207+0000","last_undegraded":"2023-11-28T11:52:09.333207+0000","last_fullsized":"2023-11-28T11:52:09.333207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:39.431994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340089+0000","last_change":"2023-11-28T11:52:08.925569+0000","last_active":"2023-11-28T11:52:09.340089+0000","last_peered":"2023-11-28T11:52:09.340089+0000","last_clean":"2023-11-28T11:52:09.340089+0000","last_became_active":"2023-11-28T11:52:08.924620+0000","last_became_peered":"2023-11-28T11:52:08.924620+0000","last_unstale":"2023-11-28T11:52:09.340089+0000","last_undegraded":"2023-11-28T11:52:09.340089+0000","last_fullsized":"2023-11-28T11:52:09.340089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:12.665527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331141+0000","last_change":"2023-11-28T11:52:09.325960+0000","last_active":"2023-11-28T11:52:09.331141+0000","last_peered":"2023-11-28T11:52:09.331141+0000","last_clean":"2023-11-28T11:52:09.331141+0000","last_became_active":"2023-11-28T11:52:09.325595+0000","last_became_peered":"2023-11-28T11:52:09.325595+0000","last_unstale":"2023-11-28T11:52:09.331141+0000","last_undegraded":"2023-11-28T11:52:09.331141+0000","last_fullsized":"2023-11-28T11:52:09.331141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:08.310622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336507+0000","last_change":"2023-11-28T11:52:08.929278+0000","last_active":"2023-11-28T11:52:09.336507+0000","last_peered":"2023-11-28T11:52:09.336507+0000","last_clean":"2023-11-28T11:52:09.336507+0000","last_became_active":"2023-11-28T11:52:08.929017+0000","last_became_peered":"2023-11-28T11:52:08.929017+0000","last_unstale":"2023-11-28T11:52:09.336507+0000","last_undegraded":"2023-11-28T11:52:09.336507+0000","last_fullsized":"2023-11-28T11:52:09.336507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:59.504922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331667+0000","last_change":"2023-11-28T11:52:09.322801+0000","last_active":"2023-11-28T11:52:09.331667+0000","last_peered":"2023-11-28T11:52:09.331667+0000","last_clean":"2023-11-28T11:52:09.331667+0000","last_became_active":"2023-11-28T11:52:09.322452+0000","last_became_peered":"2023-11-28T11:52:09.322452+0000","last_unstale":"2023-11-28T11:52:09.331667+0000","last_undegraded":"2023-11-28T11:52:09.331667+0000","last_fullsized":"2023-11-28T11:52:09.331667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:37.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339687+0000","last_change":"2023-11-28T11:52:08.924811+0000","last_active":"2023-11-28T11:52:09.339687+0000","last_peered":"2023-11-28T11:52:09.339687+0000","last_clean":"2023-11-28T11:52:09.339687+0000","last_became_active":"2023-11-28T11:52:08.924369+0000","last_became_peered":"2023-11-28T11:52:08.924369+0000","last_unstale":"2023-11-28T11:52:09.339687+0000","last_undegraded":"2023-11-28T11:52:09.339687+0000","last_fullsized":"2023-11-28T11:52:09.339687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:09.947154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342253+0000","last_change":"2023-11-28T11:52:09.338331+0000","last_active":"2023-11-28T11:52:09.342253+0000","last_peered":"2023-11-28T11:52:09.342253+0000","last_clean":"2023-11-28T11:52:09.342253+0000","last_became_active":"2023-11-28T11:52:09.337836+0000","last_became_peered":"2023-11-28T11:52:09.337836+0000","last_unstale":"2023-11-28T11:52:09.342253+0000","last_undegraded":"2023-11-28T11:52:09.342253+0000","last_fullsized":"2023-11-28T11:52:09.342253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:15:30.861236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.319067+0000","last_change":"2023-11-28T11:52:08.932248+0000","last_active":"2023-11-28T11:52:09.319067+0000","last_peered":"2023-11-28T11:52:09.319067+0000","last_clean":"2023-11-28T11:52:09.319067+0000","last_became_active":"2023-11-28T11:52:08.931999+0000","last_became_peered":"2023-11-28T11:52:08.931999+0000","last_unstale":"2023-11-28T11:52:09.319067+0000","last_undegraded":"2023-11-28T11:52:09.319067+0000","last_fullsized":"2023-11-28T11:52:09.319067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:58.426375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331382+0000","last_change":"2023-11-28T11:52:09.324608+0000","last_active":"2023-11-28T11:52:09.331382+0000","last_peered":"2023-11-28T11:52:09.331382+0000","last_clean":"2023-11-28T11:52:09.331382+0000","last_became_active":"2023-11-28T11:52:09.323879+0000","last_became_peered":"2023-11-28T11:52:09.323879+0000","last_unstale":"2023-11-28T11:52:09.331382+0000","last_undegraded":"2023-11-28T11:52:09.331382+0000","last_fullsized":"2023-11-28T11:52:09.331382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:37.353038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331554+0000","last_change":"2023-11-28T11:52:08.924100+0000","last_active":"2023-11-28T11:52:09.331554+0000","last_peered":"2023-11-28T11:52:09.331554+0000","last_clean":"2023-11-28T11:52:09.331554+0000","last_became_active":"2023-11-28T11:52:08.923842+0000","last_became_peered":"2023-11-28T11:52:08.923842+0000","last_unstale":"2023-11-28T11:52:09.331554+0000","last_undegraded":"2023-11-28T11:52:09.331554+0000","last_fullsized":"2023-11-28T11:52:09.331554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:04:25.465376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336000+0000","last_change":"2023-11-28T11:52:09.334120+0000","last_active":"2023-11-28T11:52:09.336000+0000","last_peered":"2023-11-28T11:52:09.336000+0000","last_clean":"2023-11-28T11:52:09.336000+0000","last_became_active":"2023-11-28T11:52:09.333568+0000","last_became_peered":"2023-11-28T11:52:09.333568+0000","last_unstale":"2023-11-28T11:52:09.336000+0000","last_undegraded":"2023-11-28T11:52:09.336000+0000","last_fullsized":"2023-11-28T11:52:09.336000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:59.430172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.314234+0000","last_change":"2023-11-28T11:52:08.928140+0000","last_active":"2023-11-28T11:52:09.314234+0000","last_peered":"2023-11-28T11:52:09.314234+0000","last_clean":"2023-11-28T11:52:09.314234+0000","last_became_active":"2023-11-28T11:52:08.927884+0000","last_became_peered":"2023-11-28T11:52:08.927884+0000","last_unstale":"2023-11-28T11:52:09.314234+0000","last_undegraded":"2023-11-28T11:52:09.314234+0000","last_fullsized":"2023-11-28T11:52:09.314234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:30.605334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.338142+0000","last_change":"2023-11-28T11:52:09.330674+0000","last_active":"2023-11-28T11:52:09.338142+0000","last_peered":"2023-11-28T11:52:09.338142+0000","last_clean":"2023-11-28T11:52:09.338142+0000","last_became_active":"2023-11-28T11:52:09.330224+0000","last_became_peered":"2023-11-28T11:52:09.330224+0000","last_unstale":"2023-11-28T11:52:09.338142+0000","last_undegraded":"2023-11-28T11:52:09.338142+0000","last_fullsized":"2023-11-28T11:52:09.338142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:08.317570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.313594+0000","last_change":"2023-11-28T11:52:08.932768+0000","last_active":"2023-11-28T11:52:09.313594+0000","last_peered":"2023-11-28T11:52:09.313594+0000","last_clean":"2023-11-28T11:52:09.313594+0000","last_became_active":"2023-11-28T11:52:08.932532+0000","last_became_peered":"2023-11-28T11:52:08.932532+0000","last_unstale":"2023-11-28T11:52:09.313594+0000","last_undegraded":"2023-11-28T11:52:09.313594+0000","last_fullsized":"2023-11-28T11:52:09.313594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:45.372266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325929+0000","last_change":"2023-11-28T11:52:09.325114+0000","last_active":"2023-11-28T11:52:09.325929+0000","last_peered":"2023-11-28T11:52:09.325929+0000","last_clean":"2023-11-28T11:52:09.325929+0000","last_became_active":"2023-11-28T11:52:09.324899+0000","last_became_peered":"2023-11-28T11:52:09.324899+0000","last_unstale":"2023-11-28T11:52:09.325929+0000","last_undegraded":"2023-11-28T11:52:09.325929+0000","last_fullsized":"2023-11-28T11:52:09.325929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:11.583230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341743+0000","last_change":"2023-11-28T11:52:08.929659+0000","last_active":"2023-11-28T11:52:09.341743+0000","last_peered":"2023-11-28T11:52:09.341743+0000","last_clean":"2023-11-28T11:52:09.341743+0000","last_became_active":"2023-11-28T11:52:08.929381+0000","last_became_peered":"2023-11-28T11:52:08.929381+0000","last_unstale":"2023-11-28T11:52:09.341743+0000","last_undegraded":"2023-11-28T11:52:09.341743+0000","last_fullsized":"2023-11-28T11:52:09.341743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:09.947154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342203+0000","last_change":"2023-11-28T11:52:09.337334+0000","last_active":"2023-11-28T11:52:09.342203+0000","last_peered":"2023-11-28T11:52:09.342203+0000","last_clean":"2023-11-28T11:52:09.342203+0000","last_became_active":"2023-11-28T11:52:09.334701+0000","last_became_peered":"2023-11-28T11:52:09.334701+0000","last_unstale":"2023-11-28T11:52:09.342203+0000","last_undegraded":"2023-11-28T11:52:09.342203+0000","last_fullsized":"2023-11-28T11:52:09.342203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:15.770358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342267+0000","last_change":"2023-11-28T11:52:08.920344+0000","last_active":"2023-11-28T11:52:09.342267+0000","last_peered":"2023-11-28T11:52:09.342267+0000","last_clean":"2023-11-28T11:52:09.342267+0000","last_became_active":"2023-11-28T11:52:08.920047+0000","last_became_peered":"2023-11-28T11:52:08.920047+0000","last_unstale":"2023-11-28T11:52:09.342267+0000","last_undegraded":"2023-11-28T11:52:09.342267+0000","last_fullsized":"2023-11-28T11:52:09.342267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:00.881119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339099+0000","last_change":"2023-11-28T11:52:09.338105+0000","last_active":"2023-11-28T11:52:09.339099+0000","last_peered":"2023-11-28T11:52:09.339099+0000","last_clean":"2023-11-28T11:52:09.339099+0000","last_became_active":"2023-11-28T11:52:09.337872+0000","last_became_peered":"2023-11-28T11:52:09.337872+0000","last_unstale":"2023-11-28T11:52:09.339099+0000","last_undegraded":"2023-11-28T11:52:09.339099+0000","last_fullsized":"2023-11-28T11:52:09.339099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:52.959939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331833+0000","last_change":"2023-11-28T11:52:08.925900+0000","last_active":"2023-11-28T11:52:09.331833+0000","last_peered":"2023-11-28T11:52:09.331833+0000","last_clean":"2023-11-28T11:52:09.331833+0000","last_became_active":"2023-11-28T11:52:08.925587+0000","last_became_peered":"2023-11-28T11:52:08.925587+0000","last_unstale":"2023-11-28T11:52:09.331833+0000","last_undegraded":"2023-11-28T11:52:09.331833+0000","last_fullsized":"2023-11-28T11:52:09.331833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:24:04.371969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332524+0000","last_change":"2023-11-28T11:52:09.329775+0000","last_active":"2023-11-28T11:52:09.332524+0000","last_peered":"2023-11-28T11:52:09.332524+0000","last_clean":"2023-11-28T11:52:09.332524+0000","last_became_active":"2023-11-28T11:52:09.329552+0000","last_became_peered":"2023-11-28T11:52:09.329552+0000","last_unstale":"2023-11-28T11:52:09.332524+0000","last_undegraded":"2023-11-28T11:52:09.332524+0000","last_fullsized":"2023-11-28T11:52:09.332524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:37.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318864+0000","last_change":"2023-11-28T11:52:08.930190+0000","last_active":"2023-11-28T11:52:09.318864+0000","last_peered":"2023-11-28T11:52:09.318864+0000","last_clean":"2023-11-28T11:52:09.318864+0000","last_became_active":"2023-11-28T11:52:08.929960+0000","last_became_peered":"2023-11-28T11:52:08.929960+0000","last_unstale":"2023-11-28T11:52:09.318864+0000","last_undegraded":"2023-11-28T11:52:09.318864+0000","last_fullsized":"2023-11-28T11:52:09.318864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:07.605606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.326552+0000","last_change":"2023-11-28T11:52:09.322755+0000","last_active":"2023-11-28T11:52:09.326552+0000","last_peered":"2023-11-28T11:52:09.326552+0000","last_clean":"2023-11-28T11:52:09.326552+0000","last_became_active":"2023-11-28T11:52:09.322466+0000","last_became_peered":"2023-11-28T11:52:09.322466+0000","last_unstale":"2023-11-28T11:52:09.326552+0000","last_undegraded":"2023-11-28T11:52:09.326552+0000","last_fullsized":"2023-11-28T11:52:09.326552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:45.192293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.322867+0000","last_change":"2023-11-28T11:52:08.922456+0000","last_active":"2023-11-28T11:52:09.322867+0000","last_peered":"2023-11-28T11:52:09.322867+0000","last_clean":"2023-11-28T11:52:09.322867+0000","last_became_active":"2023-11-28T11:52:08.921816+0000","last_became_peered":"2023-11-28T11:52:08.921816+0000","last_unstale":"2023-11-28T11:52:09.322867+0000","last_undegraded":"2023-11-28T11:52:09.322867+0000","last_fullsized":"2023-11-28T11:52:09.322867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:09.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332930+0000","last_change":"2023-11-28T11:52:09.331225+0000","last_active":"2023-11-28T11:52:09.332930+0000","last_peered":"2023-11-28T11:52:09.332930+0000","last_clean":"2023-11-28T11:52:09.332930+0000","last_became_active":"2023-11-28T11:52:09.329336+0000","last_became_peered":"2023-11-28T11:52:09.329336+0000","last_unstale":"2023-11-28T11:52:09.332930+0000","last_undegraded":"2023-11-28T11:52:09.332930+0000","last_fullsized":"2023-11-28T11:52:09.332930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:15.770358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342462+0000","last_change":"2023-11-28T11:52:08.924814+0000","last_active":"2023-11-28T11:52:09.342462+0000","last_peered":"2023-11-28T11:52:09.342462+0000","last_clean":"2023-11-28T11:52:09.342462+0000","last_became_active":"2023-11-28T11:52:08.924022+0000","last_became_peered":"2023-11-28T11:52:08.924022+0000","last_unstale":"2023-11-28T11:52:09.342462+0000","last_undegraded":"2023-11-28T11:52:09.342462+0000","last_fullsized":"2023-11-28T11:52:09.342462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:22.988671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332422+0000","last_change":"2023-11-28T11:52:09.329415+0000","last_active":"2023-11-28T11:52:09.332422+0000","last_peered":"2023-11-28T11:52:09.332422+0000","last_clean":"2023-11-28T11:52:09.332422+0000","last_became_active":"2023-11-28T11:52:09.329142+0000","last_became_peered":"2023-11-28T11:52:09.329142+0000","last_unstale":"2023-11-28T11:52:09.332422+0000","last_undegraded":"2023-11-28T11:52:09.332422+0000","last_fullsized":"2023-11-28T11:52:09.332422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:28.093634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339899+0000","last_change":"2023-11-28T11:52:08.931119+0000","last_active":"2023-11-28T11:52:09.339899+0000","last_peered":"2023-11-28T11:52:09.339899+0000","last_clean":"2023-11-28T11:52:09.339899+0000","last_became_active":"2023-11-28T11:52:08.930784+0000","last_became_peered":"2023-11-28T11:52:08.930784+0000","last_unstale":"2023-11-28T11:52:09.339899+0000","last_undegraded":"2023-11-28T11:52:09.339899+0000","last_fullsized":"2023-11-28T11:52:09.339899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:14.568044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336861+0000","last_change":"2023-11-28T11:52:09.325823+0000","last_active":"2023-11-28T11:52:09.336861+0000","last_peered":"2023-11-28T11:52:09.336861+0000","last_clean":"2023-11-28T11:52:09.336861+0000","last_became_active":"2023-11-28T11:52:09.325535+0000","last_became_peered":"2023-11-28T11:52:09.325535+0000","last_unstale":"2023-11-28T11:52:09.336861+0000","last_undegraded":"2023-11-28T11:52:09.336861+0000","last_fullsized":"2023-11-28T11:52:09.336861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:47.116622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342118+0000","last_change":"2023-11-28T11:52:08.932222+0000","last_active":"2023-11-28T11:52:09.342118+0000","last_peered":"2023-11-28T11:52:09.342118+0000","last_clean":"2023-11-28T11:52:09.342118+0000","last_became_active":"2023-11-28T11:52:08.931924+0000","last_became_peered":"2023-11-28T11:52:08.931924+0000","last_unstale":"2023-11-28T11:52:09.342118+0000","last_undegraded":"2023-11-28T11:52:09.342118+0000","last_fullsized":"2023-11-28T11:52:09.342118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:07.313773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340096+0000","last_change":"2023-11-28T11:52:09.335246+0000","last_active":"2023-11-28T11:52:09.340096+0000","last_peered":"2023-11-28T11:52:09.340096+0000","last_clean":"2023-11-28T11:52:09.340096+0000","last_became_active":"2023-11-28T11:52:09.333182+0000","last_became_peered":"2023-11-28T11:52:09.333182+0000","last_unstale":"2023-11-28T11:52:09.340096+0000","last_undegraded":"2023-11-28T11:52:09.340096+0000","last_fullsized":"2023-11-28T11:52:09.340096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:02.930317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.326845+0000","last_change":"2023-11-28T11:52:08.923778+0000","last_active":"2023-11-28T11:52:09.326845+0000","last_peered":"2023-11-28T11:52:09.326845+0000","last_clean":"2023-11-28T11:52:09.326845+0000","last_became_active":"2023-11-28T11:52:08.923475+0000","last_became_peered":"2023-11-28T11:52:08.923475+0000","last_unstale":"2023-11-28T11:52:09.326845+0000","last_undegraded":"2023-11-28T11:52:09.326845+0000","last_fullsized":"2023-11-28T11:52:09.326845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:08.312527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331863+0000","last_change":"2023-11-28T11:52:09.330617+0000","last_active":"2023-11-28T11:52:09.331863+0000","last_peered":"2023-11-28T11:52:09.331863+0000","last_clean":"2023-11-28T11:52:09.331863+0000","last_became_active":"2023-11-28T11:52:09.330361+0000","last_became_peered":"2023-11-28T11:52:09.330361+0000","last_unstale":"2023-11-28T11:52:09.331863+0000","last_undegraded":"2023-11-28T11:52:09.331863+0000","last_fullsized":"2023-11-28T11:52:09.331863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:12.817724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.319019+0000","last_change":"2023-11-28T11:52:08.928740+0000","last_active":"2023-11-28T11:52:09.319019+0000","last_peered":"2023-11-28T11:52:09.319019+0000","last_clean":"2023-11-28T11:52:09.319019+0000","last_became_active":"2023-11-28T11:52:08.928423+0000","last_became_peered":"2023-11-28T11:52:08.928423+0000","last_unstale":"2023-11-28T11:52:09.319019+0000","last_undegraded":"2023-11-28T11:52:09.319019+0000","last_fullsized":"2023-11-28T11:52:09.319019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:39.788967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325862+0000","last_change":"2023-11-28T11:52:09.323445+0000","last_active":"2023-11-28T11:52:09.325862+0000","last_peered":"2023-11-28T11:52:09.325862+0000","last_clean":"2023-11-28T11:52:09.325862+0000","last_became_active":"2023-11-28T11:52:09.323144+0000","last_became_peered":"2023-11-28T11:52:09.323144+0000","last_unstale":"2023-11-28T11:52:09.325862+0000","last_undegraded":"2023-11-28T11:52:09.325862+0000","last_fullsized":"2023-11-28T11:52:09.325862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:17.206689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.327948+0000","last_change":"2023-11-28T11:52:08.928152+0000","last_active":"2023-11-28T11:52:09.327948+0000","last_peered":"2023-11-28T11:52:09.327948+0000","last_clean":"2023-11-28T11:52:09.327948+0000","last_became_active":"2023-11-28T11:52:08.927632+0000","last_became_peered":"2023-11-28T11:52:08.927632+0000","last_unstale":"2023-11-28T11:52:09.327948+0000","last_undegraded":"2023-11-28T11:52:09.327948+0000","last_fullsized":"2023-11-28T11:52:09.327948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:45.307617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332729+0000","last_change":"2023-11-28T11:52:09.331354+0000","last_active":"2023-11-28T11:52:09.332729+0000","last_peered":"2023-11-28T11:52:09.332729+0000","last_clean":"2023-11-28T11:52:09.332729+0000","last_became_active":"2023-11-28T11:52:09.331000+0000","last_became_peered":"2023-11-28T11:52:09.331000+0000","last_unstale":"2023-11-28T11:52:09.332729+0000","last_undegraded":"2023-11-28T11:52:09.332729+0000","last_fullsized":"2023-11-28T11:52:09.332729+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:43.540356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.319245+0000","last_change":"2023-11-28T11:52:08.923193+0000","last_active":"2023-11-28T11:52:09.319245+0000","last_peered":"2023-11-28T11:52:09.319245+0000","last_clean":"2023-11-28T11:52:09.319245+0000","last_became_active":"2023-11-28T11:52:08.922911+0000","last_became_peered":"2023-11-28T11:52:08.922911+0000","last_unstale":"2023-11-28T11:52:09.319245+0000","last_undegraded":"2023-11-28T11:52:09.319245+0000","last_fullsized":"2023-11-28T11:52:09.319245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:35:34.323198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.334404+0000","last_change":"2023-11-28T11:52:09.328214+0000","last_active":"2023-11-28T11:52:09.334404+0000","last_peered":"2023-11-28T11:52:09.334404+0000","last_clean":"2023-11-28T11:52:09.334404+0000","last_became_active":"2023-11-28T11:52:09.327914+0000","last_became_peered":"2023-11-28T11:52:09.327914+0000","last_unstale":"2023-11-28T11:52:09.334404+0000","last_undegraded":"2023-11-28T11:52:09.334404+0000","last_fullsized":"2023-11-28T11:52:09.334404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:48.462626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.316970+0000","last_change":"2023-11-28T11:52:08.927752+0000","last_active":"2023-11-28T11:52:09.316970+0000","last_peered":"2023-11-28T11:52:09.316970+0000","last_clean":"2023-11-28T11:52:09.316970+0000","last_became_active":"2023-11-28T11:52:08.925448+0000","last_became_peered":"2023-11-28T11:52:08.925448+0000","last_unstale":"2023-11-28T11:52:09.316970+0000","last_undegraded":"2023-11-28T11:52:09.316970+0000","last_fullsized":"2023-11-28T11:52:09.316970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:25.950662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331398+0000","last_change":"2023-11-28T11:52:09.324780+0000","last_active":"2023-11-28T11:52:09.331398+0000","last_peered":"2023-11-28T11:52:09.331398+0000","last_clean":"2023-11-28T11:52:09.331398+0000","last_became_active":"2023-11-28T11:52:09.324346+0000","last_became_peered":"2023-11-28T11:52:09.324346+0000","last_unstale":"2023-11-28T11:52:09.331398+0000","last_undegraded":"2023-11-28T11:52:09.331398+0000","last_fullsized":"2023-11-28T11:52:09.331398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:02.930317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318905+0000","last_change":"2023-11-28T11:52:08.927876+0000","last_active":"2023-11-28T11:52:09.318905+0000","last_peered":"2023-11-28T11:52:09.318905+0000","last_clean":"2023-11-28T11:52:09.318905+0000","last_became_active":"2023-11-28T11:52:08.927604+0000","last_became_peered":"2023-11-28T11:52:08.927604+0000","last_unstale":"2023-11-28T11:52:09.318905+0000","last_undegraded":"2023-11-28T11:52:09.318905+0000","last_fullsized":"2023-11-28T11:52:09.318905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:20.697622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339533+0000","last_change":"2023-11-28T11:52:09.332710+0000","last_active":"2023-11-28T11:52:09.339533+0000","last_peered":"2023-11-28T11:52:09.339533+0000","last_clean":"2023-11-28T11:52:09.339533+0000","last_became_active":"2023-11-28T11:52:09.332397+0000","last_became_peered":"2023-11-28T11:52:09.332397+0000","last_unstale":"2023-11-28T11:52:09.339533+0000","last_undegraded":"2023-11-28T11:52:09.339533+0000","last_fullsized":"2023-11-28T11:52:09.339533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:11.323168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.324621+0000","last_change":"2023-11-28T11:52:08.924941+0000","last_active":"2023-11-28T11:52:09.324621+0000","last_peered":"2023-11-28T11:52:09.324621+0000","last_clean":"2023-11-28T11:52:09.324621+0000","last_became_active":"2023-11-28T11:52:08.924610+0000","last_became_peered":"2023-11-28T11:52:08.924610+0000","last_unstale":"2023-11-28T11:52:09.324621+0000","last_undegraded":"2023-11-28T11:52:09.324621+0000","last_fullsized":"2023-11-28T11:52:09.324621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:08.015178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.919325+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:08.919325+0000","last_peered":"2023-11-28T11:52:08.919325+0000","last_clean":"2023-11-28T11:52:08.919325+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:08.919325+0000","last_undegraded":"2023-11-28T11:52:08.919325+0000","last_fullsized":"2023-11-28T11:52:08.919325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.338262+0000","last_change":"2023-11-28T11:52:09.331560+0000","last_active":"2023-11-28T11:52:09.338262+0000","last_peered":"2023-11-28T11:52:09.338262+0000","last_clean":"2023-11-28T11:52:09.338262+0000","last_became_active":"2023-11-28T11:52:09.331288+0000","last_became_peered":"2023-11-28T11:52:09.331288+0000","last_unstale":"2023-11-28T11:52:09.338262+0000","last_undegraded":"2023-11-28T11:52:09.338262+0000","last_fullsized":"2023-11-28T11:52:09.338262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:13.669324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.313484+0000","last_change":"2023-11-28T11:52:08.925951+0000","last_active":"2023-11-28T11:52:09.313484+0000","last_peered":"2023-11-28T11:52:09.313484+0000","last_clean":"2023-11-28T11:52:09.313484+0000","last_became_active":"2023-11-28T11:52:08.925671+0000","last_became_peered":"2023-11-28T11:52:08.925671+0000","last_unstale":"2023-11-28T11:52:09.313484+0000","last_undegraded":"2023-11-28T11:52:09.313484+0000","last_fullsized":"2023-11-28T11:52:09.313484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:23:46.240102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.915053+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:08.915053+0000","last_peered":"2023-11-28T11:52:08.915053+0000","last_clean":"2023-11-28T11:52:08.915053+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:08.915053+0000","last_undegraded":"2023-11-28T11:52:08.915053+0000","last_fullsized":"2023-11-28T11:52:08.915053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.329241+0000","last_change":"2023-11-28T11:52:09.328111+0000","last_active":"2023-11-28T11:52:09.329241+0000","last_peered":"2023-11-28T11:52:09.329241+0000","last_clean":"2023-11-28T11:52:09.329241+0000","last_became_active":"2023-11-28T11:52:09.327510+0000","last_became_peered":"2023-11-28T11:52:09.327510+0000","last_unstale":"2023-11-28T11:52:09.329241+0000","last_undegraded":"2023-11-28T11:52:09.329241+0000","last_fullsized":"2023-11-28T11:52:09.329241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:24:05.375766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341526+0000","last_change":"2023-11-28T11:52:08.935650+0000","last_active":"2023-11-28T11:52:09.341526+0000","last_peered":"2023-11-28T11:52:09.341526+0000","last_clean":"2023-11-28T11:52:09.341526+0000","last_became_active":"2023-11-28T11:52:08.935397+0000","last_became_peered":"2023-11-28T11:52:08.935397+0000","last_unstale":"2023-11-28T11:52:09.341526+0000","last_undegraded":"2023-11-28T11:52:09.341526+0000","last_fullsized":"2023-11-28T11:52:09.341526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:25.950662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.924782+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:08.924782+0000","last_peered":"2023-11-28T11:52:08.924782+0000","last_clean":"2023-11-28T11:52:08.924782+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:08.924782+0000","last_undegraded":"2023-11-28T11:52:08.924782+0000","last_fullsized":"2023-11-28T11:52:08.924782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":173,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.916129+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:08.916129+0000","last_peered":"2023-11-28T11:52:08.916129+0000","last_clean":"2023-11-28T11:52:08.916129+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:08.916129+0000","last_undegraded":"2023-11-28T11:52:08.916129+0000","last_fullsized":"2023-11-28T11:52:08.916129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.337735+0000","last_change":"2023-11-28T11:52:09.330481+0000","last_active":"2023-11-28T11:52:09.337735+0000","last_peered":"2023-11-28T11:52:09.337735+0000","last_clean":"2023-11-28T11:52:09.337735+0000","last_became_active":"2023-11-28T11:52:09.329861+0000","last_became_peered":"2023-11-28T11:52:09.329861+0000","last_unstale":"2023-11-28T11:52:09.337735+0000","last_undegraded":"2023-11-28T11:52:09.337735+0000","last_fullsized":"2023-11-28T11:52:09.337735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:09.018975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.320659+0000","last_change":"2023-11-28T11:52:08.931516+0000","last_active":"2023-11-28T11:52:09.320659+0000","last_peered":"2023-11-28T11:52:09.320659+0000","last_clean":"2023-11-28T11:52:09.320659+0000","last_became_active":"2023-11-28T11:52:08.931196+0000","last_became_peered":"2023-11-28T11:52:08.931196+0000","last_unstale":"2023-11-28T11:52:09.320659+0000","last_undegraded":"2023-11-28T11:52:09.320659+0000","last_fullsized":"2023-11-28T11:52:09.320659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:32.421694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.916914+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:08.916914+0000","last_peered":"2023-11-28T11:52:08.916914+0000","last_clean":"2023-11-28T11:52:08.916914+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:08.916914+0000","last_undegraded":"2023-11-28T11:52:08.916914+0000","last_fullsized":"2023-11-28T11:52:08.916914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336713+0000","last_change":"2023-11-28T11:52:09.332575+0000","last_active":"2023-11-28T11:52:09.336713+0000","last_peered":"2023-11-28T11:52:09.336713+0000","last_clean":"2023-11-28T11:52:09.336713+0000","last_became_active":"2023-11-28T11:52:09.332281+0000","last_became_peered":"2023-11-28T11:52:09.332281+0000","last_unstale":"2023-11-28T11:52:09.336713+0000","last_undegraded":"2023-11-28T11:52:09.336713+0000","last_fullsized":"2023-11-28T11:52:09.336713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:45.192293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318100+0000","last_change":"2023-11-28T11:52:08.924332+0000","last_active":"2023-11-28T11:52:09.318100+0000","last_peered":"2023-11-28T11:52:09.318100+0000","last_clean":"2023-11-28T11:52:09.318100+0000","last_became_active":"2023-11-28T11:52:08.924044+0000","last_became_peered":"2023-11-28T11:52:08.924044+0000","last_unstale":"2023-11-28T11:52:09.318100+0000","last_undegraded":"2023-11-28T11:52:09.318100+0000","last_fullsized":"2023-11-28T11:52:09.318100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:14.568044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.916032+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:08.916032+0000","last_peered":"2023-11-28T11:52:08.916032+0000","last_clean":"2023-11-28T11:52:08.916032+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:08.916032+0000","last_undegraded":"2023-11-28T11:52:08.916032+0000","last_fullsized":"2023-11-28T11:52:08.916032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331060+0000","last_change":"2023-11-28T11:52:09.324928+0000","last_active":"2023-11-28T11:52:09.331060+0000","last_peered":"2023-11-28T11:52:09.331060+0000","last_clean":"2023-11-28T11:52:09.331060+0000","last_became_active":"2023-11-28T11:52:09.324669+0000","last_became_peered":"2023-11-28T11:52:09.324669+0000","last_unstale":"2023-11-28T11:52:09.331060+0000","last_undegraded":"2023-11-28T11:52:09.331060+0000","last_fullsized":"2023-11-28T11:52:09.331060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:39.402399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.322822+0000","last_change":"2023-11-28T11:52:08.927915+0000","last_active":"2023-11-28T11:52:09.322822+0000","last_peered":"2023-11-28T11:52:09.322822+0000","last_clean":"2023-11-28T11:52:09.322822+0000","last_became_active":"2023-11-28T11:52:08.927540+0000","last_became_peered":"2023-11-28T11:52:08.927540+0000","last_unstale":"2023-11-28T11:52:09.322822+0000","last_undegraded":"2023-11-28T11:52:09.322822+0000","last_fullsized":"2023-11-28T11:52:09.322822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:19.385321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.930842+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:08.930842+0000","last_peered":"2023-11-28T11:52:08.930842+0000","last_clean":"2023-11-28T11:52:08.930842+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:08.930842+0000","last_undegraded":"2023-11-28T11:52:08.930842+0000","last_fullsized":"2023-11-28T11:52:08.930842+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340743+0000","last_change":"2023-11-28T11:52:09.328204+0000","last_active":"2023-11-28T11:52:09.340743+0000","last_peered":"2023-11-28T11:52:09.340743+0000","last_clean":"2023-11-28T11:52:09.340743+0000","last_became_active":"2023-11-28T11:52:09.327600+0000","last_became_peered":"2023-11-28T11:52:09.327600+0000","last_unstale":"2023-11-28T11:52:09.340743+0000","last_undegraded":"2023-11-28T11:52:09.340743+0000","last_fullsized":"2023-11-28T11:52:09.340743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.578877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.322807+0000","last_change":"2023-11-28T11:52:08.926089+0000","last_active":"2023-11-28T11:52:09.322807+0000","last_peered":"2023-11-28T11:52:09.322807+0000","last_clean":"2023-11-28T11:52:09.322807+0000","last_became_active":"2023-11-28T11:52:08.925829+0000","last_became_peered":"2023-11-28T11:52:08.925829+0000","last_unstale":"2023-11-28T11:52:09.322807+0000","last_undegraded":"2023-11-28T11:52:09.322807+0000","last_fullsized":"2023-11-28T11:52:09.322807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:14.868584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.917482+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:08.917482+0000","last_peered":"2023-11-28T11:52:08.917482+0000","last_clean":"2023-11-28T11:52:08.917482+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:08.917482+0000","last_undegraded":"2023-11-28T11:52:08.917482+0000","last_fullsized":"2023-11-28T11:52:08.917482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.333550+0000","last_change":"2023-11-28T11:52:09.332444+0000","last_active":"2023-11-28T11:52:09.333550+0000","last_peered":"2023-11-28T11:52:09.333550+0000","last_clean":"2023-11-28T11:52:09.333550+0000","last_became_active":"2023-11-28T11:52:09.332237+0000","last_became_peered":"2023-11-28T11:52:09.332237+0000","last_unstale":"2023-11-28T11:52:09.333550+0000","last_undegraded":"2023-11-28T11:52:09.333550+0000","last_fullsized":"2023-11-28T11:52:09.333550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:32.972348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318500+0000","last_change":"2023-11-28T11:52:08.925427+0000","last_active":"2023-11-28T11:52:09.318500+0000","last_peered":"2023-11-28T11:52:09.318500+0000","last_clean":"2023-11-28T11:52:09.318500+0000","last_became_active":"2023-11-28T11:52:08.925096+0000","last_became_peered":"2023-11-28T11:52:08.925096+0000","last_unstale":"2023-11-28T11:52:09.318500+0000","last_undegraded":"2023-11-28T11:52:09.318500+0000","last_fullsized":"2023-11-28T11:52:09.318500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:58:23.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.917479+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:08.917479+0000","last_peered":"2023-11-28T11:52:08.917479+0000","last_clean":"2023-11-28T11:52:08.917479+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:08.917479+0000","last_undegraded":"2023-11-28T11:52:08.917479+0000","last_fullsized":"2023-11-28T11:52:08.917479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9372,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362468,"statfs":{"total":96636764160,"available":96627167232,"internally_reserved":0,"allocated":937984,"data_stored":725465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8228,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363612,"statfs":{"total":96636764160,"available":96628338688,"internally_reserved":0,"allocated":356352,"data_stored":141725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9308,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362532,"statfs":{"total":96636764160,"available":96627232768,"internally_reserved":0,"allocated":937984,"data_stored":725465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8368,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363472,"statfs":{"total":96636764160,"available":96628195328,"internally_reserved":0,"allocated":368640,"data_stored":147901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":372736,"data_stored":148923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.317},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.41299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.65300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.68999999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.60899999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8448,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363392,"statfs":{"total":96636764160,"available":96628113408,"internally_reserved":0,"allocated":385024,"data_stored":150435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8504,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363336,"statfs":{"total":96636764160,"available":96628056064,"internally_reserved":0,"allocated":376832,"data_stored":149441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8372,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363468,"statfs":{"total":96636764160,"available":96628191232,"internally_reserved":0,"allocated":372736,"data_stored":147979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:13.391 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:13.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.546+0000 7fded4749700 1 Processor -- start 2023-11-28T11:52:13.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.546+0000 7fded4749700 1 -- start start 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.546+0000 7fded4749700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.546+0000 7fded4749700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdecc160540 0x7fdecc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded4749700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc15d060 con 0x7fdecc071040 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded4749700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc15d340 con 0x7fdecc160540 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded4749700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc15d620 con 0x7fdecc06dee0 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdecc160540 0x7fdecc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdecc160540 0x7fdecc1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36440/0 (socket says 172.21.15.144:36440) 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 -- 172.21.15.144:0/2524076626 learned_addr learned my addr 172.21.15.144:0/2524076626 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded1ce4700 1 --2- 172.21.15.144:0/2524076626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 -- 172.21.15.144:0/2524076626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 msgr2=0x7fdecc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 --2- 172.21.15.144:0/2524076626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 -- 172.21.15.144:0/2524076626 >> v1:172.21.15.177:6789/0 conn(0x7fdecc071040 legacy=0x7fdecc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 -- 172.21.15.144:0/2524076626 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc15d900 con 0x7fdecc160540 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded2ce6700 1 --2- 172.21.15.144:0/2524076626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc160540 0x7fdecc1609c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009e50 tx=0x7fdec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cc3a654ced4a87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded14e3700 1 -- 172.21.15.144:0/2524076626 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400e070 con 0x7fdecc160540 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded14e3700 1 -- 172.21.15.144:0/2524076626 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec4003a20 con 0x7fdecc160540 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.547+0000 7fded14e3700 1 -- 172.21.15.144:0/2524076626 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4003dd0 con 0x7fdecc160540 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 -- 172.21.15.144:0/2524076626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc160540 msgr2=0x7fdecc1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 --2- 172.21.15.144:0/2524076626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc160540 0x7fdecc1609c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009e50 tx=0x7fdec400b330 comp rx=0 tx=0).stop 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 -- 172.21.15.144:0/2524076626 shutdown_connections 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 --2- 172.21.15.144:0/2524076626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc160540 0x7fdecc1609c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 --2- 172.21.15.144:0/2524076626 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 -- 172.21.15.144:0/2524076626 >> 172.21.15.144:0/2524076626 conn(0x7fdecc0d0000 msgr2=0x7fdecc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:13.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 -- 172.21.15.144:0/2524076626 shutdown_connections 2023-11-28T11:52:13.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.548+0000 7fded4749700 1 -- 172.21.15.144:0/2524076626 wait complete. 2023-11-28T11:52:13.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.549+0000 7fded4749700 1 Processor -- start 2023-11-28T11:52:13.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.549+0000 7fded4749700 1 -- start start 2023-11-28T11:52:13.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded4749700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded4749700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 0x7fdecc156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded4749700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdecc160540 0x7fdecc0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded4749700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc159310 con 0x7fdecc071040 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded4749700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0c4850 con 0x7fdecc160540 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded4749700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc0c4b30 con 0x7fdecc06dee0 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 0x7fdecc156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 0x7fdecc156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36444/0 (socket says 172.21.15.144:36444) 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 -- 172.21.15.144:0/1898069189 learned_addr learned my addr 172.21.15.144:0/1898069189 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded2ce6700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdecc160540 0x7fdecc0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded24e5700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 -- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 msgr2=0x7fdecc158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 -- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdecc160540 msgr2=0x7fdecc0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdecc160540 0x7fdecc0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 -- 172.21.15.144:0/1898069189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4009b30 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.550+0000 7fded1ce4700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 0x7fdecc156890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c7f0 tx=0x7fdec800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.551+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8011800 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.551+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8011960 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.551+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc15ccd0 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.551+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8011cf0 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.551+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc06e870 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.552+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc06a270 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.552+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdec8021070 con 0x7fdecc071040 2023-11-28T11:52:13.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.552+0000 7fdebeffd700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdeb00682f0 0x7fdeb006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.553+0000 7fded24e5700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdeb00682f0 0x7fdeb006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.553+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdec8085a40 con 0x7fdecc071040 2023-11-28T11:52:13.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.553+0000 7fded24e5700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdeb00682f0 0x7fdeb006a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdecc15f910 tx=0x7fdec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.557+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdec80522a0 con 0x7fdecc071040 2023-11-28T11:52:13.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.735+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdecc0720a0 con 0x7fdeb00682f0 2023-11-28T11:52:13.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.742+0000 7fdebeffd700 1 -- 172.21.15.144:0/1898069189 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290061 (secure 0 0 0) 0x7fdecc0720a0 con 0x7fdeb00682f0 2023-11-28T11:52:13.746 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:13.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdeb00682f0 msgr2=0x7fdeb006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdeb00682f0 0x7fdeb006a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdecc15f910 tx=0x7fdec0008040 comp rx=0 tx=0).stop 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 msgr2=0x7fdecc156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 0x7fdecc156890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c7f0 tx=0x7fdec800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 shutdown_connections 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdeb00682f0 0x7fdeb006a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdecc160540 0x7fdecc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdecc06dee0 0x7fdecc158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.745+0000 7fded4749700 1 --2- 172.21.15.144:0/1898069189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdecc071040 0x7fdecc156890 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.746+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 >> 172.21.15.144:0/1898069189 conn(0x7fdecc0d0000 msgr2=0x7fdecc0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.746+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 shutdown_connections 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.746+0000 7fded4749700 1 -- 172.21.15.144:0/1898069189 wait complete. 2023-11-28T11:52:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:13.758 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-11-28T11:52:12.494421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":68844,"kb_used_data":4012,"kb_used_omap":0,"kb_used_meta":64768,"kb_avail":754905876,"statfs":{"total":773094113280,"available":773023617024,"internally_reserved":0,"allocated":4108288,"data_stored":2337334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66322432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.590254"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332498+0000","last_change":"2023-11-28T11:52:09.329046+0000","last_active":"2023-11-28T11:52:09.332498+0000","last_peered":"2023-11-28T11:52:09.332498+0000","last_clean":"2023-11-28T11:52:09.332498+0000","last_became_active":"2023-11-28T11:52:09.328834+0000","last_became_peered":"2023-11-28T11:52:09.328834+0000","last_unstale":"2023-11-28T11:52:09.332498+0000","last_undegraded":"2023-11-28T11:52:09.332498+0000","last_fullsized":"2023-11-28T11:52:09.332498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:47.116622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331041+0000","last_change":"2023-11-28T11:52:09.324674+0000","last_active":"2023-11-28T11:52:09.331041+0000","last_peered":"2023-11-28T11:52:09.331041+0000","last_clean":"2023-11-28T11:52:09.331041+0000","last_became_active":"2023-11-28T11:52:09.324411+0000","last_became_peered":"2023-11-28T11:52:09.324411+0000","last_unstale":"2023-11-28T11:52:09.331041+0000","last_undegraded":"2023-11-28T11:52:09.331041+0000","last_fullsized":"2023-11-28T11:52:09.331041+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:11.323168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336171+0000","last_change":"2023-11-28T11:52:09.335158+0000","last_active":"2023-11-28T11:52:09.336171+0000","last_peered":"2023-11-28T11:52:09.336171+0000","last_clean":"2023-11-28T11:52:09.336171+0000","last_became_active":"2023-11-28T11:52:09.334912+0000","last_became_peered":"2023-11-28T11:52:09.334912+0000","last_unstale":"2023-11-28T11:52:09.336171+0000","last_undegraded":"2023-11-28T11:52:09.336171+0000","last_fullsized":"2023-11-28T11:52:09.336171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:07.358461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336021+0000","last_change":"2023-11-28T11:52:09.334226+0000","last_active":"2023-11-28T11:52:09.336021+0000","last_peered":"2023-11-28T11:52:09.336021+0000","last_clean":"2023-11-28T11:52:09.336021+0000","last_became_active":"2023-11-28T11:52:09.333892+0000","last_became_peered":"2023-11-28T11:52:09.333892+0000","last_unstale":"2023-11-28T11:52:09.336021+0000","last_undegraded":"2023-11-28T11:52:09.336021+0000","last_fullsized":"2023-11-28T11:52:09.336021+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:56.578314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.335919+0000","last_change":"2023-11-28T11:52:09.333770+0000","last_active":"2023-11-28T11:52:09.335919+0000","last_peered":"2023-11-28T11:52:09.335919+0000","last_clean":"2023-11-28T11:52:09.335919+0000","last_became_active":"2023-11-28T11:52:09.333502+0000","last_became_peered":"2023-11-28T11:52:09.333502+0000","last_unstale":"2023-11-28T11:52:09.335919+0000","last_undegraded":"2023-11-28T11:52:09.335919+0000","last_fullsized":"2023-11-28T11:52:09.335919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:40.792764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341345+0000","last_change":"2023-11-28T11:52:09.330238+0000","last_active":"2023-11-28T11:52:09.341345+0000","last_peered":"2023-11-28T11:52:09.341345+0000","last_clean":"2023-11-28T11:52:09.341345+0000","last_became_active":"2023-11-28T11:52:09.330022+0000","last_became_peered":"2023-11-28T11:52:09.330022+0000","last_unstale":"2023-11-28T11:52:09.341345+0000","last_undegraded":"2023-11-28T11:52:09.341345+0000","last_fullsized":"2023-11-28T11:52:09.341345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:29.214687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336803+0000","last_change":"2023-11-28T11:52:09.332627+0000","last_active":"2023-11-28T11:52:09.336803+0000","last_peered":"2023-11-28T11:52:09.336803+0000","last_clean":"2023-11-28T11:52:09.336803+0000","last_became_active":"2023-11-28T11:52:09.332334+0000","last_became_peered":"2023-11-28T11:52:09.332334+0000","last_unstale":"2023-11-28T11:52:09.336803+0000","last_undegraded":"2023-11-28T11:52:09.336803+0000","last_fullsized":"2023-11-28T11:52:09.336803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:11.323168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.330949+0000","last_change":"2023-11-28T11:52:09.323546+0000","last_active":"2023-11-28T11:52:09.330949+0000","last_peered":"2023-11-28T11:52:09.330949+0000","last_clean":"2023-11-28T11:52:09.330949+0000","last_became_active":"2023-11-28T11:52:09.323304+0000","last_became_peered":"2023-11-28T11:52:09.323304+0000","last_unstale":"2023-11-28T11:52:09.330949+0000","last_undegraded":"2023-11-28T11:52:09.330949+0000","last_fullsized":"2023-11-28T11:52:09.330949+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:15:30.861236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336175+0000","last_change":"2023-11-28T11:52:09.335162+0000","last_active":"2023-11-28T11:52:09.336175+0000","last_peered":"2023-11-28T11:52:09.336175+0000","last_clean":"2023-11-28T11:52:09.336175+0000","last_became_active":"2023-11-28T11:52:09.334875+0000","last_became_peered":"2023-11-28T11:52:09.334875+0000","last_unstale":"2023-11-28T11:52:09.336175+0000","last_undegraded":"2023-11-28T11:52:09.336175+0000","last_fullsized":"2023-11-28T11:52:09.336175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:20:59.977365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336859+0000","last_change":"2023-11-28T11:52:09.330989+0000","last_active":"2023-11-28T11:52:09.336859+0000","last_peered":"2023-11-28T11:52:09.336859+0000","last_clean":"2023-11-28T11:52:09.336859+0000","last_became_active":"2023-11-28T11:52:09.330760+0000","last_became_peered":"2023-11-28T11:52:09.330760+0000","last_unstale":"2023-11-28T11:52:09.336859+0000","last_undegraded":"2023-11-28T11:52:09.336859+0000","last_fullsized":"2023-11-28T11:52:09.336859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:28.093634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336918+0000","last_change":"2023-11-28T11:52:09.333179+0000","last_active":"2023-11-28T11:52:09.336918+0000","last_peered":"2023-11-28T11:52:09.336918+0000","last_clean":"2023-11-28T11:52:09.336918+0000","last_became_active":"2023-11-28T11:52:09.332920+0000","last_became_peered":"2023-11-28T11:52:09.332920+0000","last_unstale":"2023-11-28T11:52:09.336918+0000","last_undegraded":"2023-11-28T11:52:09.336918+0000","last_fullsized":"2023-11-28T11:52:09.336918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:24.177480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325926+0000","last_change":"2023-11-28T11:52:09.323600+0000","last_active":"2023-11-28T11:52:09.325926+0000","last_peered":"2023-11-28T11:52:09.325926+0000","last_clean":"2023-11-28T11:52:09.325926+0000","last_became_active":"2023-11-28T11:52:09.323332+0000","last_became_peered":"2023-11-28T11:52:09.323332+0000","last_unstale":"2023-11-28T11:52:09.325926+0000","last_undegraded":"2023-11-28T11:52:09.325926+0000","last_fullsized":"2023-11-28T11:52:09.325926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:04:37.612996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.337105+0000","last_change":"2023-11-28T11:52:09.330628+0000","last_active":"2023-11-28T11:52:09.337105+0000","last_peered":"2023-11-28T11:52:09.337105+0000","last_clean":"2023-11-28T11:52:09.337105+0000","last_became_active":"2023-11-28T11:52:09.330332+0000","last_became_peered":"2023-11-28T11:52:09.330332+0000","last_unstale":"2023-11-28T11:52:09.337105+0000","last_undegraded":"2023-11-28T11:52:09.337105+0000","last_fullsized":"2023-11-28T11:52:09.337105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:39.431994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340190+0000","last_change":"2023-11-28T11:52:09.334264+0000","last_active":"2023-11-28T11:52:09.340190+0000","last_peered":"2023-11-28T11:52:09.340190+0000","last_clean":"2023-11-28T11:52:09.340190+0000","last_became_active":"2023-11-28T11:52:09.333315+0000","last_became_peered":"2023-11-28T11:52:09.333315+0000","last_unstale":"2023-11-28T11:52:09.340190+0000","last_undegraded":"2023-11-28T11:52:09.340190+0000","last_fullsized":"2023-11-28T11:52:09.340190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:28.093634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.326603+0000","last_change":"2023-11-28T11:52:09.320288+0000","last_active":"2023-11-28T11:52:09.326603+0000","last_peered":"2023-11-28T11:52:09.326603+0000","last_clean":"2023-11-28T11:52:09.326603+0000","last_became_active":"2023-11-28T11:52:09.319977+0000","last_became_peered":"2023-11-28T11:52:09.319977+0000","last_unstale":"2023-11-28T11:52:09.326603+0000","last_undegraded":"2023-11-28T11:52:09.326603+0000","last_fullsized":"2023-11-28T11:52:09.326603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:07:09.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336483+0000","last_change":"2023-11-28T11:52:09.327570+0000","last_active":"2023-11-28T11:52:09.336483+0000","last_peered":"2023-11-28T11:52:09.336483+0000","last_clean":"2023-11-28T11:52:09.336483+0000","last_became_active":"2023-11-28T11:52:09.327037+0000","last_became_peered":"2023-11-28T11:52:09.327037+0000","last_unstale":"2023-11-28T11:52:09.336483+0000","last_undegraded":"2023-11-28T11:52:09.336483+0000","last_fullsized":"2023-11-28T11:52:09.336483+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:47.116622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336660+0000","last_change":"2023-11-28T11:52:09.327560+0000","last_active":"2023-11-28T11:52:09.336660+0000","last_peered":"2023-11-28T11:52:09.336660+0000","last_clean":"2023-11-28T11:52:09.336660+0000","last_became_active":"2023-11-28T11:52:09.327251+0000","last_became_peered":"2023-11-28T11:52:09.327251+0000","last_unstale":"2023-11-28T11:52:09.336660+0000","last_undegraded":"2023-11-28T11:52:09.336660+0000","last_fullsized":"2023-11-28T11:52:09.336660+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:39.431994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.344792+0000","last_change":"2023-11-28T11:52:09.340087+0000","last_active":"2023-11-28T11:52:09.344792+0000","last_peered":"2023-11-28T11:52:09.344792+0000","last_clean":"2023-11-28T11:52:09.344792+0000","last_became_active":"2023-11-28T11:52:09.339836+0000","last_became_peered":"2023-11-28T11:52:09.339836+0000","last_unstale":"2023-11-28T11:52:09.344792+0000","last_undegraded":"2023-11-28T11:52:09.344792+0000","last_fullsized":"2023-11-28T11:52:09.344792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.578877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.329491+0000","last_change":"2023-11-28T11:52:09.327409+0000","last_active":"2023-11-28T11:52:09.329491+0000","last_peered":"2023-11-28T11:52:09.329491+0000","last_clean":"2023-11-28T11:52:09.329491+0000","last_became_active":"2023-11-28T11:52:09.327137+0000","last_became_peered":"2023-11-28T11:52:09.327137+0000","last_unstale":"2023-11-28T11:52:09.329491+0000","last_undegraded":"2023-11-28T11:52:09.329491+0000","last_fullsized":"2023-11-28T11:52:09.329491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:00.508719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342107+0000","last_change":"2023-11-28T11:52:09.338731+0000","last_active":"2023-11-28T11:52:09.342107+0000","last_peered":"2023-11-28T11:52:09.342107+0000","last_clean":"2023-11-28T11:52:09.342107+0000","last_became_active":"2023-11-28T11:52:09.338412+0000","last_became_peered":"2023-11-28T11:52:09.338412+0000","last_unstale":"2023-11-28T11:52:09.342107+0000","last_undegraded":"2023-11-28T11:52:09.342107+0000","last_fullsized":"2023-11-28T11:52:09.342107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:46.376063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.334499+0000","last_change":"2023-11-28T11:52:09.325697+0000","last_active":"2023-11-28T11:52:09.334499+0000","last_peered":"2023-11-28T11:52:09.334499+0000","last_clean":"2023-11-28T11:52:09.334499+0000","last_became_active":"2023-11-28T11:52:09.325415+0000","last_became_peered":"2023-11-28T11:52:09.325415+0000","last_unstale":"2023-11-28T11:52:09.334499+0000","last_undegraded":"2023-11-28T11:52:09.334499+0000","last_fullsized":"2023-11-28T11:52:09.334499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:17.206689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340213+0000","last_change":"2023-11-28T11:52:09.336362+0000","last_active":"2023-11-28T11:52:09.340213+0000","last_peered":"2023-11-28T11:52:09.340213+0000","last_clean":"2023-11-28T11:52:09.340213+0000","last_became_active":"2023-11-28T11:52:09.336071+0000","last_became_peered":"2023-11-28T11:52:09.336071+0000","last_unstale":"2023-11-28T11:52:09.340213+0000","last_undegraded":"2023-11-28T11:52:09.340213+0000","last_fullsized":"2023-11-28T11:52:09.340213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:37.353038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.330225+0000","last_change":"2023-11-28T11:52:09.324610+0000","last_active":"2023-11-28T11:52:09.330225+0000","last_peered":"2023-11-28T11:52:09.330225+0000","last_clean":"2023-11-28T11:52:09.330225+0000","last_became_active":"2023-11-28T11:52:09.323323+0000","last_became_peered":"2023-11-28T11:52:09.323323+0000","last_unstale":"2023-11-28T11:52:09.330225+0000","last_undegraded":"2023-11-28T11:52:09.330225+0000","last_fullsized":"2023-11-28T11:52:09.330225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:08:24.177480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325971+0000","last_change":"2023-11-28T11:52:09.324187+0000","last_active":"2023-11-28T11:52:09.325971+0000","last_peered":"2023-11-28T11:52:09.325971+0000","last_clean":"2023-11-28T11:52:09.325971+0000","last_became_active":"2023-11-28T11:52:09.323880+0000","last_became_peered":"2023-11-28T11:52:09.323880+0000","last_unstale":"2023-11-28T11:52:09.325971+0000","last_undegraded":"2023-11-28T11:52:09.325971+0000","last_fullsized":"2023-11-28T11:52:09.325971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:08.609403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331768+0000","last_change":"2023-11-28T11:52:09.327794+0000","last_active":"2023-11-28T11:52:09.331768+0000","last_peered":"2023-11-28T11:52:09.331768+0000","last_clean":"2023-11-28T11:52:09.331768+0000","last_became_active":"2023-11-28T11:52:09.327512+0000","last_became_peered":"2023-11-28T11:52:09.327512+0000","last_unstale":"2023-11-28T11:52:09.331768+0000","last_undegraded":"2023-11-28T11:52:09.331768+0000","last_fullsized":"2023-11-28T11:52:09.331768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:15.571841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341412+0000","last_change":"2023-11-28T11:52:09.335760+0000","last_active":"2023-11-28T11:52:09.341412+0000","last_peered":"2023-11-28T11:52:09.341412+0000","last_clean":"2023-11-28T11:52:09.341412+0000","last_became_active":"2023-11-28T11:52:09.335479+0000","last_became_peered":"2023-11-28T11:52:09.335479+0000","last_unstale":"2023-11-28T11:52:09.341412+0000","last_undegraded":"2023-11-28T11:52:09.341412+0000","last_fullsized":"2023-11-28T11:52:09.341412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:31.609131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331774+0000","last_change":"2023-11-28T11:52:09.329173+0000","last_active":"2023-11-28T11:52:09.331774+0000","last_peered":"2023-11-28T11:52:09.331774+0000","last_clean":"2023-11-28T11:52:09.331774+0000","last_became_active":"2023-11-28T11:52:09.328857+0000","last_became_peered":"2023-11-28T11:52:09.328857+0000","last_unstale":"2023-11-28T11:52:09.331774+0000","last_undegraded":"2023-11-28T11:52:09.331774+0000","last_fullsized":"2023-11-28T11:52:09.331774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:10.950951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336776+0000","last_change":"2023-11-28T11:52:09.332719+0000","last_active":"2023-11-28T11:52:09.336776+0000","last_peered":"2023-11-28T11:52:09.336776+0000","last_clean":"2023-11-28T11:52:09.336776+0000","last_became_active":"2023-11-28T11:52:09.332434+0000","last_became_peered":"2023-11-28T11:52:09.332434+0000","last_unstale":"2023-11-28T11:52:09.336776+0000","last_undegraded":"2023-11-28T11:52:09.336776+0000","last_fullsized":"2023-11-28T11:52:09.336776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:09.018975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331159+0000","last_change":"2023-11-28T11:52:09.326773+0000","last_active":"2023-11-28T11:52:09.331159+0000","last_peered":"2023-11-28T11:52:09.331159+0000","last_clean":"2023-11-28T11:52:09.331159+0000","last_became_active":"2023-11-28T11:52:09.326452+0000","last_became_peered":"2023-11-28T11:52:09.326452+0000","last_unstale":"2023-11-28T11:52:09.331159+0000","last_undegraded":"2023-11-28T11:52:09.331159+0000","last_fullsized":"2023-11-28T11:52:09.331159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:29.214687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.333590+0000","last_change":"2023-11-28T11:52:09.331101+0000","last_active":"2023-11-28T11:52:09.333590+0000","last_peered":"2023-11-28T11:52:09.333590+0000","last_clean":"2023-11-28T11:52:09.333590+0000","last_became_active":"2023-11-28T11:52:09.330801+0000","last_became_peered":"2023-11-28T11:52:09.330801+0000","last_unstale":"2023-11-28T11:52:09.333590+0000","last_undegraded":"2023-11-28T11:52:09.333590+0000","last_fullsized":"2023-11-28T11:52:09.333590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:26.954458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.330246+0000","last_change":"2023-11-28T11:52:09.324800+0000","last_active":"2023-11-28T11:52:09.330246+0000","last_peered":"2023-11-28T11:52:09.330246+0000","last_clean":"2023-11-28T11:52:09.330246+0000","last_became_active":"2023-11-28T11:52:09.323685+0000","last_became_peered":"2023-11-28T11:52:09.323685+0000","last_unstale":"2023-11-28T11:52:09.330246+0000","last_undegraded":"2023-11-28T11:52:09.330246+0000","last_fullsized":"2023-11-28T11:52:09.330246+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:48.462626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.324238+0000","last_change":"2023-11-28T11:52:09.322618+0000","last_active":"2023-11-28T11:52:09.324238+0000","last_peered":"2023-11-28T11:52:09.324238+0000","last_clean":"2023-11-28T11:52:09.324238+0000","last_became_active":"2023-11-28T11:52:09.322358+0000","last_became_peered":"2023-11-28T11:52:09.322358+0000","last_unstale":"2023-11-28T11:52:09.324238+0000","last_undegraded":"2023-11-28T11:52:09.324238+0000","last_fullsized":"2023-11-28T11:52:09.324238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:01.884916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336695+0000","last_change":"2023-11-28T11:52:09.328737+0000","last_active":"2023-11-28T11:52:09.336695+0000","last_peered":"2023-11-28T11:52:09.336695+0000","last_clean":"2023-11-28T11:52:09.336695+0000","last_became_active":"2023-11-28T11:52:09.328289+0000","last_became_peered":"2023-11-28T11:52:09.328289+0000","last_unstale":"2023-11-28T11:52:09.336695+0000","last_undegraded":"2023-11-28T11:52:09.336695+0000","last_fullsized":"2023-11-28T11:52:09.336695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:29:02.011479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342529+0000","last_change":"2023-11-28T11:52:08.922237+0000","last_active":"2023-11-28T11:52:09.342529+0000","last_peered":"2023-11-28T11:52:09.342529+0000","last_clean":"2023-11-28T11:52:09.342529+0000","last_became_active":"2023-11-28T11:52:08.921926+0000","last_became_peered":"2023-11-28T11:52:08.921926+0000","last_unstale":"2023-11-28T11:52:09.342529+0000","last_undegraded":"2023-11-28T11:52:09.342529+0000","last_fullsized":"2023-11-28T11:52:09.342529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:23:54.348920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341296+0000","last_change":"2023-11-28T11:52:09.329043+0000","last_active":"2023-11-28T11:52:09.341296+0000","last_peered":"2023-11-28T11:52:09.341296+0000","last_clean":"2023-11-28T11:52:09.341296+0000","last_became_active":"2023-11-28T11:52:09.328613+0000","last_became_peered":"2023-11-28T11:52:09.328613+0000","last_unstale":"2023-11-28T11:52:09.341296+0000","last_undegraded":"2023-11-28T11:52:09.341296+0000","last_fullsized":"2023-11-28T11:52:09.341296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:23:47.243899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336885+0000","last_change":"2023-11-28T11:52:08.920434+0000","last_active":"2023-11-28T11:52:09.336885+0000","last_peered":"2023-11-28T11:52:09.336885+0000","last_clean":"2023-11-28T11:52:09.336885+0000","last_became_active":"2023-11-28T11:52:08.920162+0000","last_became_peered":"2023-11-28T11:52:08.920162+0000","last_unstale":"2023-11-28T11:52:09.336885+0000","last_undegraded":"2023-11-28T11:52:09.336885+0000","last_fullsized":"2023-11-28T11:52:09.336885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:16:36.400378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339981+0000","last_change":"2023-11-28T11:52:09.334508+0000","last_active":"2023-11-28T11:52:09.339981+0000","last_peered":"2023-11-28T11:52:09.339981+0000","last_clean":"2023-11-28T11:52:09.339981+0000","last_became_active":"2023-11-28T11:52:09.334181+0000","last_became_peered":"2023-11-28T11:52:09.334181+0000","last_unstale":"2023-11-28T11:52:09.339981+0000","last_undegraded":"2023-11-28T11:52:09.339981+0000","last_fullsized":"2023-11-28T11:52:09.339981+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:37.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339743+0000","last_change":"2023-11-28T11:52:08.928738+0000","last_active":"2023-11-28T11:52:09.339743+0000","last_peered":"2023-11-28T11:52:09.339743+0000","last_clean":"2023-11-28T11:52:09.339743+0000","last_became_active":"2023-11-28T11:52:08.928428+0000","last_became_peered":"2023-11-28T11:52:08.928428+0000","last_unstale":"2023-11-28T11:52:09.339743+0000","last_undegraded":"2023-11-28T11:52:09.339743+0000","last_fullsized":"2023-11-28T11:52:09.339743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:55.574517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.333207+0000","last_change":"2023-11-28T11:52:09.332189+0000","last_active":"2023-11-28T11:52:09.333207+0000","last_peered":"2023-11-28T11:52:09.333207+0000","last_clean":"2023-11-28T11:52:09.333207+0000","last_became_active":"2023-11-28T11:52:09.331957+0000","last_became_peered":"2023-11-28T11:52:09.331957+0000","last_unstale":"2023-11-28T11:52:09.333207+0000","last_undegraded":"2023-11-28T11:52:09.333207+0000","last_fullsized":"2023-11-28T11:52:09.333207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:39.431994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340089+0000","last_change":"2023-11-28T11:52:08.925569+0000","last_active":"2023-11-28T11:52:09.340089+0000","last_peered":"2023-11-28T11:52:09.340089+0000","last_clean":"2023-11-28T11:52:09.340089+0000","last_became_active":"2023-11-28T11:52:08.924620+0000","last_became_peered":"2023-11-28T11:52:08.924620+0000","last_unstale":"2023-11-28T11:52:09.340089+0000","last_undegraded":"2023-11-28T11:52:09.340089+0000","last_fullsized":"2023-11-28T11:52:09.340089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:12.665527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331141+0000","last_change":"2023-11-28T11:52:09.325960+0000","last_active":"2023-11-28T11:52:09.331141+0000","last_peered":"2023-11-28T11:52:09.331141+0000","last_clean":"2023-11-28T11:52:09.331141+0000","last_became_active":"2023-11-28T11:52:09.325595+0000","last_became_peered":"2023-11-28T11:52:09.325595+0000","last_unstale":"2023-11-28T11:52:09.331141+0000","last_undegraded":"2023-11-28T11:52:09.331141+0000","last_fullsized":"2023-11-28T11:52:09.331141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:08.310622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336507+0000","last_change":"2023-11-28T11:52:08.929278+0000","last_active":"2023-11-28T11:52:09.336507+0000","last_peered":"2023-11-28T11:52:09.336507+0000","last_clean":"2023-11-28T11:52:09.336507+0000","last_became_active":"2023-11-28T11:52:08.929017+0000","last_became_peered":"2023-11-28T11:52:08.929017+0000","last_unstale":"2023-11-28T11:52:09.336507+0000","last_undegraded":"2023-11-28T11:52:09.336507+0000","last_fullsized":"2023-11-28T11:52:09.336507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:59.504922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331667+0000","last_change":"2023-11-28T11:52:09.322801+0000","last_active":"2023-11-28T11:52:09.331667+0000","last_peered":"2023-11-28T11:52:09.331667+0000","last_clean":"2023-11-28T11:52:09.331667+0000","last_became_active":"2023-11-28T11:52:09.322452+0000","last_became_peered":"2023-11-28T11:52:09.322452+0000","last_unstale":"2023-11-28T11:52:09.331667+0000","last_undegraded":"2023-11-28T11:52:09.331667+0000","last_fullsized":"2023-11-28T11:52:09.331667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:37.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339687+0000","last_change":"2023-11-28T11:52:08.924811+0000","last_active":"2023-11-28T11:52:09.339687+0000","last_peered":"2023-11-28T11:52:09.339687+0000","last_clean":"2023-11-28T11:52:09.339687+0000","last_became_active":"2023-11-28T11:52:08.924369+0000","last_became_peered":"2023-11-28T11:52:08.924369+0000","last_unstale":"2023-11-28T11:52:09.339687+0000","last_undegraded":"2023-11-28T11:52:09.339687+0000","last_fullsized":"2023-11-28T11:52:09.339687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:09.947154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342253+0000","last_change":"2023-11-28T11:52:09.338331+0000","last_active":"2023-11-28T11:52:09.342253+0000","last_peered":"2023-11-28T11:52:09.342253+0000","last_clean":"2023-11-28T11:52:09.342253+0000","last_became_active":"2023-11-28T11:52:09.337836+0000","last_became_peered":"2023-11-28T11:52:09.337836+0000","last_unstale":"2023-11-28T11:52:09.342253+0000","last_undegraded":"2023-11-28T11:52:09.342253+0000","last_fullsized":"2023-11-28T11:52:09.342253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:15:30.861236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.319067+0000","last_change":"2023-11-28T11:52:08.932248+0000","last_active":"2023-11-28T11:52:09.319067+0000","last_peered":"2023-11-28T11:52:09.319067+0000","last_clean":"2023-11-28T11:52:09.319067+0000","last_became_active":"2023-11-28T11:52:08.931999+0000","last_became_peered":"2023-11-28T11:52:08.931999+0000","last_unstale":"2023-11-28T11:52:09.319067+0000","last_undegraded":"2023-11-28T11:52:09.319067+0000","last_fullsized":"2023-11-28T11:52:09.319067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:58.426375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331382+0000","last_change":"2023-11-28T11:52:09.324608+0000","last_active":"2023-11-28T11:52:09.331382+0000","last_peered":"2023-11-28T11:52:09.331382+0000","last_clean":"2023-11-28T11:52:09.331382+0000","last_became_active":"2023-11-28T11:52:09.323879+0000","last_became_peered":"2023-11-28T11:52:09.323879+0000","last_unstale":"2023-11-28T11:52:09.331382+0000","last_undegraded":"2023-11-28T11:52:09.331382+0000","last_fullsized":"2023-11-28T11:52:09.331382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:18:37.353038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331554+0000","last_change":"2023-11-28T11:52:08.924100+0000","last_active":"2023-11-28T11:52:09.331554+0000","last_peered":"2023-11-28T11:52:09.331554+0000","last_clean":"2023-11-28T11:52:09.331554+0000","last_became_active":"2023-11-28T11:52:08.923842+0000","last_became_peered":"2023-11-28T11:52:08.923842+0000","last_unstale":"2023-11-28T11:52:09.331554+0000","last_undegraded":"2023-11-28T11:52:09.331554+0000","last_fullsized":"2023-11-28T11:52:09.331554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:04:25.465376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336000+0000","last_change":"2023-11-28T11:52:09.334120+0000","last_active":"2023-11-28T11:52:09.336000+0000","last_peered":"2023-11-28T11:52:09.336000+0000","last_clean":"2023-11-28T11:52:09.336000+0000","last_became_active":"2023-11-28T11:52:09.333568+0000","last_became_peered":"2023-11-28T11:52:09.333568+0000","last_unstale":"2023-11-28T11:52:09.336000+0000","last_undegraded":"2023-11-28T11:52:09.336000+0000","last_fullsized":"2023-11-28T11:52:09.336000+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:59.430172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.314234+0000","last_change":"2023-11-28T11:52:08.928140+0000","last_active":"2023-11-28T11:52:09.314234+0000","last_peered":"2023-11-28T11:52:09.314234+0000","last_clean":"2023-11-28T11:52:09.314234+0000","last_became_active":"2023-11-28T11:52:08.927884+0000","last_became_peered":"2023-11-28T11:52:08.927884+0000","last_unstale":"2023-11-28T11:52:09.314234+0000","last_undegraded":"2023-11-28T11:52:09.314234+0000","last_fullsized":"2023-11-28T11:52:09.314234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:09:30.605334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.338142+0000","last_change":"2023-11-28T11:52:09.330674+0000","last_active":"2023-11-28T11:52:09.338142+0000","last_peered":"2023-11-28T11:52:09.338142+0000","last_clean":"2023-11-28T11:52:09.338142+0000","last_became_active":"2023-11-28T11:52:09.330224+0000","last_became_peered":"2023-11-28T11:52:09.330224+0000","last_unstale":"2023-11-28T11:52:09.338142+0000","last_undegraded":"2023-11-28T11:52:09.338142+0000","last_fullsized":"2023-11-28T11:52:09.338142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:08.317570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.313594+0000","last_change":"2023-11-28T11:52:08.932768+0000","last_active":"2023-11-28T11:52:09.313594+0000","last_peered":"2023-11-28T11:52:09.313594+0000","last_clean":"2023-11-28T11:52:09.313594+0000","last_became_active":"2023-11-28T11:52:08.932532+0000","last_became_peered":"2023-11-28T11:52:08.932532+0000","last_unstale":"2023-11-28T11:52:09.313594+0000","last_undegraded":"2023-11-28T11:52:09.313594+0000","last_fullsized":"2023-11-28T11:52:09.313594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:45.372266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325929+0000","last_change":"2023-11-28T11:52:09.325114+0000","last_active":"2023-11-28T11:52:09.325929+0000","last_peered":"2023-11-28T11:52:09.325929+0000","last_clean":"2023-11-28T11:52:09.325929+0000","last_became_active":"2023-11-28T11:52:09.324899+0000","last_became_peered":"2023-11-28T11:52:09.324899+0000","last_unstale":"2023-11-28T11:52:09.325929+0000","last_undegraded":"2023-11-28T11:52:09.325929+0000","last_fullsized":"2023-11-28T11:52:09.325929+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:11.583230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341743+0000","last_change":"2023-11-28T11:52:08.929659+0000","last_active":"2023-11-28T11:52:09.341743+0000","last_peered":"2023-11-28T11:52:09.341743+0000","last_clean":"2023-11-28T11:52:09.341743+0000","last_became_active":"2023-11-28T11:52:08.929381+0000","last_became_peered":"2023-11-28T11:52:08.929381+0000","last_unstale":"2023-11-28T11:52:09.341743+0000","last_undegraded":"2023-11-28T11:52:09.341743+0000","last_fullsized":"2023-11-28T11:52:09.341743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:32:09.947154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342203+0000","last_change":"2023-11-28T11:52:09.337334+0000","last_active":"2023-11-28T11:52:09.342203+0000","last_peered":"2023-11-28T11:52:09.342203+0000","last_clean":"2023-11-28T11:52:09.342203+0000","last_became_active":"2023-11-28T11:52:09.334701+0000","last_became_peered":"2023-11-28T11:52:09.334701+0000","last_unstale":"2023-11-28T11:52:09.342203+0000","last_undegraded":"2023-11-28T11:52:09.342203+0000","last_fullsized":"2023-11-28T11:52:09.342203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:15.770358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342267+0000","last_change":"2023-11-28T11:52:08.920344+0000","last_active":"2023-11-28T11:52:09.342267+0000","last_peered":"2023-11-28T11:52:09.342267+0000","last_clean":"2023-11-28T11:52:09.342267+0000","last_became_active":"2023-11-28T11:52:08.920047+0000","last_became_peered":"2023-11-28T11:52:08.920047+0000","last_unstale":"2023-11-28T11:52:09.342267+0000","last_undegraded":"2023-11-28T11:52:09.342267+0000","last_fullsized":"2023-11-28T11:52:09.342267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:00.881119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339099+0000","last_change":"2023-11-28T11:52:09.338105+0000","last_active":"2023-11-28T11:52:09.339099+0000","last_peered":"2023-11-28T11:52:09.339099+0000","last_clean":"2023-11-28T11:52:09.339099+0000","last_became_active":"2023-11-28T11:52:09.337872+0000","last_became_peered":"2023-11-28T11:52:09.337872+0000","last_unstale":"2023-11-28T11:52:09.339099+0000","last_undegraded":"2023-11-28T11:52:09.339099+0000","last_fullsized":"2023-11-28T11:52:09.339099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:52.959939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331833+0000","last_change":"2023-11-28T11:52:08.925900+0000","last_active":"2023-11-28T11:52:09.331833+0000","last_peered":"2023-11-28T11:52:09.331833+0000","last_clean":"2023-11-28T11:52:09.331833+0000","last_became_active":"2023-11-28T11:52:08.925587+0000","last_became_peered":"2023-11-28T11:52:08.925587+0000","last_unstale":"2023-11-28T11:52:09.331833+0000","last_undegraded":"2023-11-28T11:52:09.331833+0000","last_fullsized":"2023-11-28T11:52:09.331833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:24:04.371969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332524+0000","last_change":"2023-11-28T11:52:09.329775+0000","last_active":"2023-11-28T11:52:09.332524+0000","last_peered":"2023-11-28T11:52:09.332524+0000","last_clean":"2023-11-28T11:52:09.332524+0000","last_became_active":"2023-11-28T11:52:09.329552+0000","last_became_peered":"2023-11-28T11:52:09.329552+0000","last_unstale":"2023-11-28T11:52:09.332524+0000","last_undegraded":"2023-11-28T11:52:09.332524+0000","last_fullsized":"2023-11-28T11:52:09.332524+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:42:37.645205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318864+0000","last_change":"2023-11-28T11:52:08.930190+0000","last_active":"2023-11-28T11:52:09.318864+0000","last_peered":"2023-11-28T11:52:09.318864+0000","last_clean":"2023-11-28T11:52:09.318864+0000","last_became_active":"2023-11-28T11:52:08.929960+0000","last_became_peered":"2023-11-28T11:52:08.929960+0000","last_unstale":"2023-11-28T11:52:09.318864+0000","last_undegraded":"2023-11-28T11:52:09.318864+0000","last_fullsized":"2023-11-28T11:52:09.318864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:07.605606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.326552+0000","last_change":"2023-11-28T11:52:09.322755+0000","last_active":"2023-11-28T11:52:09.326552+0000","last_peered":"2023-11-28T11:52:09.326552+0000","last_clean":"2023-11-28T11:52:09.326552+0000","last_became_active":"2023-11-28T11:52:09.322466+0000","last_became_peered":"2023-11-28T11:52:09.322466+0000","last_unstale":"2023-11-28T11:52:09.326552+0000","last_undegraded":"2023-11-28T11:52:09.326552+0000","last_fullsized":"2023-11-28T11:52:09.326552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:45.192293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.322867+0000","last_change":"2023-11-28T11:52:08.922456+0000","last_active":"2023-11-28T11:52:09.322867+0000","last_peered":"2023-11-28T11:52:09.322867+0000","last_clean":"2023-11-28T11:52:09.322867+0000","last_became_active":"2023-11-28T11:52:08.921816+0000","last_became_peered":"2023-11-28T11:52:08.921816+0000","last_unstale":"2023-11-28T11:52:09.322867+0000","last_undegraded":"2023-11-28T11:52:09.322867+0000","last_fullsized":"2023-11-28T11:52:09.322867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:09.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332930+0000","last_change":"2023-11-28T11:52:09.331225+0000","last_active":"2023-11-28T11:52:09.332930+0000","last_peered":"2023-11-28T11:52:09.332930+0000","last_clean":"2023-11-28T11:52:09.332930+0000","last_became_active":"2023-11-28T11:52:09.329336+0000","last_became_peered":"2023-11-28T11:52:09.329336+0000","last_unstale":"2023-11-28T11:52:09.332930+0000","last_undegraded":"2023-11-28T11:52:09.332930+0000","last_fullsized":"2023-11-28T11:52:09.332930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:15.770358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342462+0000","last_change":"2023-11-28T11:52:08.924814+0000","last_active":"2023-11-28T11:52:09.342462+0000","last_peered":"2023-11-28T11:52:09.342462+0000","last_clean":"2023-11-28T11:52:09.342462+0000","last_became_active":"2023-11-28T11:52:08.924022+0000","last_became_peered":"2023-11-28T11:52:08.924022+0000","last_unstale":"2023-11-28T11:52:09.342462+0000","last_undegraded":"2023-11-28T11:52:09.342462+0000","last_fullsized":"2023-11-28T11:52:09.342462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:58:22.988671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332422+0000","last_change":"2023-11-28T11:52:09.329415+0000","last_active":"2023-11-28T11:52:09.332422+0000","last_peered":"2023-11-28T11:52:09.332422+0000","last_clean":"2023-11-28T11:52:09.332422+0000","last_became_active":"2023-11-28T11:52:09.329142+0000","last_became_peered":"2023-11-28T11:52:09.329142+0000","last_unstale":"2023-11-28T11:52:09.332422+0000","last_undegraded":"2023-11-28T11:52:09.332422+0000","last_fullsized":"2023-11-28T11:52:09.332422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:28.093634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339899+0000","last_change":"2023-11-28T11:52:08.931119+0000","last_active":"2023-11-28T11:52:09.339899+0000","last_peered":"2023-11-28T11:52:09.339899+0000","last_clean":"2023-11-28T11:52:09.339899+0000","last_became_active":"2023-11-28T11:52:08.930784+0000","last_became_peered":"2023-11-28T11:52:08.930784+0000","last_unstale":"2023-11-28T11:52:09.339899+0000","last_undegraded":"2023-11-28T11:52:09.339899+0000","last_fullsized":"2023-11-28T11:52:09.339899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:14.568044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336861+0000","last_change":"2023-11-28T11:52:09.325823+0000","last_active":"2023-11-28T11:52:09.336861+0000","last_peered":"2023-11-28T11:52:09.336861+0000","last_clean":"2023-11-28T11:52:09.336861+0000","last_became_active":"2023-11-28T11:52:09.325535+0000","last_became_peered":"2023-11-28T11:52:09.325535+0000","last_unstale":"2023-11-28T11:52:09.336861+0000","last_undegraded":"2023-11-28T11:52:09.336861+0000","last_fullsized":"2023-11-28T11:52:09.336861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:02:47.116622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.342118+0000","last_change":"2023-11-28T11:52:08.932222+0000","last_active":"2023-11-28T11:52:09.342118+0000","last_peered":"2023-11-28T11:52:09.342118+0000","last_clean":"2023-11-28T11:52:09.342118+0000","last_became_active":"2023-11-28T11:52:08.931924+0000","last_became_peered":"2023-11-28T11:52:08.931924+0000","last_unstale":"2023-11-28T11:52:09.342118+0000","last_undegraded":"2023-11-28T11:52:09.342118+0000","last_fullsized":"2023-11-28T11:52:09.342118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:07.313773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340096+0000","last_change":"2023-11-28T11:52:09.335246+0000","last_active":"2023-11-28T11:52:09.340096+0000","last_peered":"2023-11-28T11:52:09.340096+0000","last_clean":"2023-11-28T11:52:09.340096+0000","last_became_active":"2023-11-28T11:52:09.333182+0000","last_became_peered":"2023-11-28T11:52:09.333182+0000","last_unstale":"2023-11-28T11:52:09.340096+0000","last_undegraded":"2023-11-28T11:52:09.340096+0000","last_fullsized":"2023-11-28T11:52:09.340096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:02.930317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.326845+0000","last_change":"2023-11-28T11:52:08.923778+0000","last_active":"2023-11-28T11:52:09.326845+0000","last_peered":"2023-11-28T11:52:09.326845+0000","last_clean":"2023-11-28T11:52:09.326845+0000","last_became_active":"2023-11-28T11:52:08.923475+0000","last_became_peered":"2023-11-28T11:52:08.923475+0000","last_unstale":"2023-11-28T11:52:09.326845+0000","last_undegraded":"2023-11-28T11:52:09.326845+0000","last_fullsized":"2023-11-28T11:52:09.326845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:08.312527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331863+0000","last_change":"2023-11-28T11:52:09.330617+0000","last_active":"2023-11-28T11:52:09.331863+0000","last_peered":"2023-11-28T11:52:09.331863+0000","last_clean":"2023-11-28T11:52:09.331863+0000","last_became_active":"2023-11-28T11:52:09.330361+0000","last_became_peered":"2023-11-28T11:52:09.330361+0000","last_unstale":"2023-11-28T11:52:09.331863+0000","last_undegraded":"2023-11-28T11:52:09.331863+0000","last_fullsized":"2023-11-28T11:52:09.331863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:12.817724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.319019+0000","last_change":"2023-11-28T11:52:08.928740+0000","last_active":"2023-11-28T11:52:09.319019+0000","last_peered":"2023-11-28T11:52:09.319019+0000","last_clean":"2023-11-28T11:52:09.319019+0000","last_became_active":"2023-11-28T11:52:08.928423+0000","last_became_peered":"2023-11-28T11:52:08.928423+0000","last_unstale":"2023-11-28T11:52:09.319019+0000","last_undegraded":"2023-11-28T11:52:09.319019+0000","last_fullsized":"2023-11-28T11:52:09.319019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:05:39.788967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.325862+0000","last_change":"2023-11-28T11:52:09.323445+0000","last_active":"2023-11-28T11:52:09.325862+0000","last_peered":"2023-11-28T11:52:09.325862+0000","last_clean":"2023-11-28T11:52:09.325862+0000","last_became_active":"2023-11-28T11:52:09.323144+0000","last_became_peered":"2023-11-28T11:52:09.323144+0000","last_unstale":"2023-11-28T11:52:09.325862+0000","last_undegraded":"2023-11-28T11:52:09.325862+0000","last_fullsized":"2023-11-28T11:52:09.325862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:17.206689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.327948+0000","last_change":"2023-11-28T11:52:08.928152+0000","last_active":"2023-11-28T11:52:09.327948+0000","last_peered":"2023-11-28T11:52:09.327948+0000","last_clean":"2023-11-28T11:52:09.327948+0000","last_became_active":"2023-11-28T11:52:08.927632+0000","last_became_peered":"2023-11-28T11:52:08.927632+0000","last_unstale":"2023-11-28T11:52:09.327948+0000","last_undegraded":"2023-11-28T11:52:09.327948+0000","last_fullsized":"2023-11-28T11:52:09.327948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:45.307617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.332729+0000","last_change":"2023-11-28T11:52:09.331354+0000","last_active":"2023-11-28T11:52:09.332729+0000","last_peered":"2023-11-28T11:52:09.332729+0000","last_clean":"2023-11-28T11:52:09.332729+0000","last_became_active":"2023-11-28T11:52:09.331000+0000","last_became_peered":"2023-11-28T11:52:09.331000+0000","last_unstale":"2023-11-28T11:52:09.332729+0000","last_undegraded":"2023-11-28T11:52:09.332729+0000","last_fullsized":"2023-11-28T11:52:09.332729+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:43.540356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.319245+0000","last_change":"2023-11-28T11:52:08.923193+0000","last_active":"2023-11-28T11:52:09.319245+0000","last_peered":"2023-11-28T11:52:09.319245+0000","last_clean":"2023-11-28T11:52:09.319245+0000","last_became_active":"2023-11-28T11:52:08.922911+0000","last_became_peered":"2023-11-28T11:52:08.922911+0000","last_unstale":"2023-11-28T11:52:09.319245+0000","last_undegraded":"2023-11-28T11:52:09.319245+0000","last_fullsized":"2023-11-28T11:52:09.319245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:35:34.323198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.334404+0000","last_change":"2023-11-28T11:52:09.328214+0000","last_active":"2023-11-28T11:52:09.334404+0000","last_peered":"2023-11-28T11:52:09.334404+0000","last_clean":"2023-11-28T11:52:09.334404+0000","last_became_active":"2023-11-28T11:52:09.327914+0000","last_became_peered":"2023-11-28T11:52:09.327914+0000","last_unstale":"2023-11-28T11:52:09.334404+0000","last_undegraded":"2023-11-28T11:52:09.334404+0000","last_fullsized":"2023-11-28T11:52:09.334404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:00:48.462626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.316970+0000","last_change":"2023-11-28T11:52:08.927752+0000","last_active":"2023-11-28T11:52:09.316970+0000","last_peered":"2023-11-28T11:52:09.316970+0000","last_clean":"2023-11-28T11:52:09.316970+0000","last_became_active":"2023-11-28T11:52:08.925448+0000","last_became_peered":"2023-11-28T11:52:08.925448+0000","last_unstale":"2023-11-28T11:52:09.316970+0000","last_undegraded":"2023-11-28T11:52:09.316970+0000","last_fullsized":"2023-11-28T11:52:09.316970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:25.950662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331398+0000","last_change":"2023-11-28T11:52:09.324780+0000","last_active":"2023-11-28T11:52:09.331398+0000","last_peered":"2023-11-28T11:52:09.331398+0000","last_clean":"2023-11-28T11:52:09.331398+0000","last_became_active":"2023-11-28T11:52:09.324346+0000","last_became_peered":"2023-11-28T11:52:09.324346+0000","last_unstale":"2023-11-28T11:52:09.331398+0000","last_undegraded":"2023-11-28T11:52:09.331398+0000","last_fullsized":"2023-11-28T11:52:09.331398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:19:02.930317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318905+0000","last_change":"2023-11-28T11:52:08.927876+0000","last_active":"2023-11-28T11:52:09.318905+0000","last_peered":"2023-11-28T11:52:09.318905+0000","last_clean":"2023-11-28T11:52:09.318905+0000","last_became_active":"2023-11-28T11:52:08.927604+0000","last_became_peered":"2023-11-28T11:52:08.927604+0000","last_unstale":"2023-11-28T11:52:09.318905+0000","last_undegraded":"2023-11-28T11:52:09.318905+0000","last_fullsized":"2023-11-28T11:52:09.318905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:49:20.697622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.339533+0000","last_change":"2023-11-28T11:52:09.332710+0000","last_active":"2023-11-28T11:52:09.339533+0000","last_peered":"2023-11-28T11:52:09.339533+0000","last_clean":"2023-11-28T11:52:09.339533+0000","last_became_active":"2023-11-28T11:52:09.332397+0000","last_became_peered":"2023-11-28T11:52:09.332397+0000","last_unstale":"2023-11-28T11:52:09.339533+0000","last_undegraded":"2023-11-28T11:52:09.339533+0000","last_fullsized":"2023-11-28T11:52:09.339533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:11.323168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.324621+0000","last_change":"2023-11-28T11:52:08.924941+0000","last_active":"2023-11-28T11:52:09.324621+0000","last_peered":"2023-11-28T11:52:09.324621+0000","last_clean":"2023-11-28T11:52:09.324621+0000","last_became_active":"2023-11-28T11:52:08.924610+0000","last_became_peered":"2023-11-28T11:52:08.924610+0000","last_unstale":"2023-11-28T11:52:09.324621+0000","last_undegraded":"2023-11-28T11:52:09.324621+0000","last_fullsized":"2023-11-28T11:52:09.324621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:08.015178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.919325+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:08.919325+0000","last_peered":"2023-11-28T11:52:08.919325+0000","last_clean":"2023-11-28T11:52:08.919325+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:08.919325+0000","last_undegraded":"2023-11-28T11:52:08.919325+0000","last_fullsized":"2023-11-28T11:52:08.919325+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.338262+0000","last_change":"2023-11-28T11:52:09.331560+0000","last_active":"2023-11-28T11:52:09.338262+0000","last_peered":"2023-11-28T11:52:09.338262+0000","last_clean":"2023-11-28T11:52:09.338262+0000","last_became_active":"2023-11-28T11:52:09.331288+0000","last_became_peered":"2023-11-28T11:52:09.331288+0000","last_unstale":"2023-11-28T11:52:09.338262+0000","last_undegraded":"2023-11-28T11:52:09.338262+0000","last_fullsized":"2023-11-28T11:52:09.338262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:13.669324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.313484+0000","last_change":"2023-11-28T11:52:08.925951+0000","last_active":"2023-11-28T11:52:09.313484+0000","last_peered":"2023-11-28T11:52:09.313484+0000","last_clean":"2023-11-28T11:52:09.313484+0000","last_became_active":"2023-11-28T11:52:08.925671+0000","last_became_peered":"2023-11-28T11:52:08.925671+0000","last_unstale":"2023-11-28T11:52:09.313484+0000","last_undegraded":"2023-11-28T11:52:09.313484+0000","last_fullsized":"2023-11-28T11:52:09.313484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:23:46.240102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.915053+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:08.915053+0000","last_peered":"2023-11-28T11:52:08.915053+0000","last_clean":"2023-11-28T11:52:08.915053+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:08.915053+0000","last_undegraded":"2023-11-28T11:52:08.915053+0000","last_fullsized":"2023-11-28T11:52:08.915053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.329241+0000","last_change":"2023-11-28T11:52:09.328111+0000","last_active":"2023-11-28T11:52:09.329241+0000","last_peered":"2023-11-28T11:52:09.329241+0000","last_clean":"2023-11-28T11:52:09.329241+0000","last_became_active":"2023-11-28T11:52:09.327510+0000","last_became_peered":"2023-11-28T11:52:09.327510+0000","last_unstale":"2023-11-28T11:52:09.329241+0000","last_undegraded":"2023-11-28T11:52:09.329241+0000","last_fullsized":"2023-11-28T11:52:09.329241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:24:05.375766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.341526+0000","last_change":"2023-11-28T11:52:08.935650+0000","last_active":"2023-11-28T11:52:09.341526+0000","last_peered":"2023-11-28T11:52:09.341526+0000","last_clean":"2023-11-28T11:52:09.341526+0000","last_became_active":"2023-11-28T11:52:08.935397+0000","last_became_peered":"2023-11-28T11:52:08.935397+0000","last_unstale":"2023-11-28T11:52:09.341526+0000","last_undegraded":"2023-11-28T11:52:09.341526+0000","last_fullsized":"2023-11-28T11:52:09.341526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:25.950662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.924782+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:08.924782+0000","last_peered":"2023-11-28T11:52:08.924782+0000","last_clean":"2023-11-28T11:52:08.924782+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:08.924782+0000","last_undegraded":"2023-11-28T11:52:08.924782+0000","last_fullsized":"2023-11-28T11:52:08.924782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":173,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.916129+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:08.916129+0000","last_peered":"2023-11-28T11:52:08.916129+0000","last_clean":"2023-11-28T11:52:08.916129+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:08.916129+0000","last_undegraded":"2023-11-28T11:52:08.916129+0000","last_fullsized":"2023-11-28T11:52:08.916129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.337735+0000","last_change":"2023-11-28T11:52:09.330481+0000","last_active":"2023-11-28T11:52:09.337735+0000","last_peered":"2023-11-28T11:52:09.337735+0000","last_clean":"2023-11-28T11:52:09.337735+0000","last_became_active":"2023-11-28T11:52:09.329861+0000","last_became_peered":"2023-11-28T11:52:09.329861+0000","last_unstale":"2023-11-28T11:52:09.337735+0000","last_undegraded":"2023-11-28T11:52:09.337735+0000","last_fullsized":"2023-11-28T11:52:09.337735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:09.018975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.320659+0000","last_change":"2023-11-28T11:52:08.931516+0000","last_active":"2023-11-28T11:52:09.320659+0000","last_peered":"2023-11-28T11:52:09.320659+0000","last_clean":"2023-11-28T11:52:09.320659+0000","last_became_active":"2023-11-28T11:52:08.931196+0000","last_became_peered":"2023-11-28T11:52:08.931196+0000","last_unstale":"2023-11-28T11:52:09.320659+0000","last_undegraded":"2023-11-28T11:52:09.320659+0000","last_fullsized":"2023-11-28T11:52:09.320659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:32.421694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.916914+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:08.916914+0000","last_peered":"2023-11-28T11:52:08.916914+0000","last_clean":"2023-11-28T11:52:08.916914+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:08.916914+0000","last_undegraded":"2023-11-28T11:52:08.916914+0000","last_fullsized":"2023-11-28T11:52:08.916914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.336713+0000","last_change":"2023-11-28T11:52:09.332575+0000","last_active":"2023-11-28T11:52:09.336713+0000","last_peered":"2023-11-28T11:52:09.336713+0000","last_clean":"2023-11-28T11:52:09.336713+0000","last_became_active":"2023-11-28T11:52:09.332281+0000","last_became_peered":"2023-11-28T11:52:09.332281+0000","last_unstale":"2023-11-28T11:52:09.336713+0000","last_undegraded":"2023-11-28T11:52:09.336713+0000","last_fullsized":"2023-11-28T11:52:09.336713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:45.192293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318100+0000","last_change":"2023-11-28T11:52:08.924332+0000","last_active":"2023-11-28T11:52:09.318100+0000","last_peered":"2023-11-28T11:52:09.318100+0000","last_clean":"2023-11-28T11:52:09.318100+0000","last_became_active":"2023-11-28T11:52:08.924044+0000","last_became_peered":"2023-11-28T11:52:08.924044+0000","last_unstale":"2023-11-28T11:52:09.318100+0000","last_undegraded":"2023-11-28T11:52:09.318100+0000","last_fullsized":"2023-11-28T11:52:09.318100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:14.568044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.916032+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:08.916032+0000","last_peered":"2023-11-28T11:52:08.916032+0000","last_clean":"2023-11-28T11:52:08.916032+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:08.916032+0000","last_undegraded":"2023-11-28T11:52:08.916032+0000","last_fullsized":"2023-11-28T11:52:08.916032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.331060+0000","last_change":"2023-11-28T11:52:09.324928+0000","last_active":"2023-11-28T11:52:09.331060+0000","last_peered":"2023-11-28T11:52:09.331060+0000","last_clean":"2023-11-28T11:52:09.331060+0000","last_became_active":"2023-11-28T11:52:09.324669+0000","last_became_peered":"2023-11-28T11:52:09.324669+0000","last_unstale":"2023-11-28T11:52:09.331060+0000","last_undegraded":"2023-11-28T11:52:09.331060+0000","last_fullsized":"2023-11-28T11:52:09.331060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:39.402399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.322822+0000","last_change":"2023-11-28T11:52:08.927915+0000","last_active":"2023-11-28T11:52:09.322822+0000","last_peered":"2023-11-28T11:52:09.322822+0000","last_clean":"2023-11-28T11:52:09.322822+0000","last_became_active":"2023-11-28T11:52:08.927540+0000","last_became_peered":"2023-11-28T11:52:08.927540+0000","last_unstale":"2023-11-28T11:52:09.322822+0000","last_undegraded":"2023-11-28T11:52:09.322822+0000","last_fullsized":"2023-11-28T11:52:09.322822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:19.385321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.930842+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:08.930842+0000","last_peered":"2023-11-28T11:52:08.930842+0000","last_clean":"2023-11-28T11:52:08.930842+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:08.930842+0000","last_undegraded":"2023-11-28T11:52:08.930842+0000","last_fullsized":"2023-11-28T11:52:08.930842+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.340743+0000","last_change":"2023-11-28T11:52:09.328204+0000","last_active":"2023-11-28T11:52:09.340743+0000","last_peered":"2023-11-28T11:52:09.340743+0000","last_clean":"2023-11-28T11:52:09.340743+0000","last_became_active":"2023-11-28T11:52:09.327600+0000","last_became_peered":"2023-11-28T11:52:09.327600+0000","last_unstale":"2023-11-28T11:52:09.340743+0000","last_undegraded":"2023-11-28T11:52:09.340743+0000","last_fullsized":"2023-11-28T11:52:09.340743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:38.578877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.322807+0000","last_change":"2023-11-28T11:52:08.926089+0000","last_active":"2023-11-28T11:52:09.322807+0000","last_peered":"2023-11-28T11:52:09.322807+0000","last_clean":"2023-11-28T11:52:09.322807+0000","last_became_active":"2023-11-28T11:52:08.925829+0000","last_became_peered":"2023-11-28T11:52:08.925829+0000","last_unstale":"2023-11-28T11:52:09.322807+0000","last_undegraded":"2023-11-28T11:52:09.322807+0000","last_fullsized":"2023-11-28T11:52:09.322807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:14.868584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.917482+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:08.917482+0000","last_peered":"2023-11-28T11:52:08.917482+0000","last_clean":"2023-11-28T11:52:08.917482+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:08.917482+0000","last_undegraded":"2023-11-28T11:52:08.917482+0000","last_fullsized":"2023-11-28T11:52:08.917482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.333550+0000","last_change":"2023-11-28T11:52:09.332444+0000","last_active":"2023-11-28T11:52:09.333550+0000","last_peered":"2023-11-28T11:52:09.333550+0000","last_clean":"2023-11-28T11:52:09.333550+0000","last_became_active":"2023-11-28T11:52:09.332237+0000","last_became_peered":"2023-11-28T11:52:09.332237+0000","last_unstale":"2023-11-28T11:52:09.333550+0000","last_undegraded":"2023-11-28T11:52:09.333550+0000","last_fullsized":"2023-11-28T11:52:09.333550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:08.901570+0000","last_clean_scrub_stamp":"2023-11-28T11:52:08.901570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:32.972348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T11:52:09.318500+0000","last_change":"2023-11-28T11:52:08.925427+0000","last_active":"2023-11-28T11:52:09.318500+0000","last_peered":"2023-11-28T11:52:09.318500+0000","last_clean":"2023-11-28T11:52:09.318500+0000","last_became_active":"2023-11-28T11:52:08.925096+0000","last_became_peered":"2023-11-28T11:52:08.925096+0000","last_unstale":"2023-11-28T11:52:09.318500+0000","last_undegraded":"2023-11-28T11:52:09.318500+0000","last_fullsized":"2023-11-28T11:52:09.318500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:07.897773+0000","last_clean_scrub_stamp":"2023-11-28T11:52:07.897773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:58:23.885714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-28T11:52:08.917479+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:08.917479+0000","last_peered":"2023-11-28T11:52:08.917479+0000","last_clean":"2023-11-28T11:52:08.917479+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:08.917479+0000","last_undegraded":"2023-11-28T11:52:08.917479+0000","last_fullsized":"2023-11-28T11:52:08.917479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9372,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362468,"statfs":{"total":96636764160,"available":96627167232,"internally_reserved":0,"allocated":937984,"data_stored":725465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8228,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363612,"statfs":{"total":96636764160,"available":96628338688,"internally_reserved":0,"allocated":356352,"data_stored":141725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9308,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362532,"statfs":{"total":96636764160,"available":96627232768,"internally_reserved":0,"allocated":937984,"data_stored":725465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8368,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363472,"statfs":{"total":96636764160,"available":96628195328,"internally_reserved":0,"allocated":368640,"data_stored":147901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8244,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363596,"statfs":{"total":96636764160,"available":96628322304,"internally_reserved":0,"allocated":372736,"data_stored":148923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.317},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.39300000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.41299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.65300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.68999999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.60899999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8448,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363392,"statfs":{"total":96636764160,"available":96628113408,"internally_reserved":0,"allocated":385024,"data_stored":150435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8504,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363336,"statfs":{"total":96636764160,"available":96628056064,"internally_reserved":0,"allocated":376832,"data_stored":149441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8372,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363468,"statfs":{"total":96636764160,"available":96628191232,"internally_reserved":0,"allocated":372736,"data_stored":147979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:13.763 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:52:13.763 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:52:13.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.925+0000 7f8dd9691700 1 Processor -- start 2023-11-28T11:52:13.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.925+0000 7f8dd9691700 1 -- start start 2023-11-28T11:52:13.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd9691700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8dd40b20b0 0x7f8dd40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd9691700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40b0230 con 0x7f8dd40d01b0 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd9691700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40b4570 con 0x7f8dd40d1330 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd9691700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd415cec0 con 0x7f8dd40b20b0 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd2ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8dd40d01b0 0x7f8dd40d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53162/0 (socket says 172.21.15.144:53162) 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd2ffd700 1 -- 172.21.15.144:0/2422064227 learned_addr learned my addr 172.21.15.144:0/2422064227 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd37fe700 1 --2- 172.21.15.144:0/2422064227 >> v2:172.21.15.177:3300/0 conn(0x7f8dd40b20b0 0x7f8dd40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.926+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104401676 0 0) 0x7f8dd40b0230 con 0x7f8dd40d01b0 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dbc0035b0 con 0x7f8dd40d01b0 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 634474688 0 0) 0x7f8dbc0035b0 con 0x7f8dd40d01b0 2023-11-28T11:52:13.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 >> v2:172.21.15.177:3300/0 conn(0x7f8dd40b20b0 msgr2=0x7f8dd40b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 --2- 172.21.15.144:0/2422064227 >> v2:172.21.15.177:3300/0 conn(0x7f8dd40b20b0 0x7f8dd40b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 >> v1:172.21.15.177:6789/0 conn(0x7f8dd40d1330 legacy=0x7f8dd40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40b4850 con 0x7f8dd40d01b0 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f8dc8001670 con 0x7f8dd40d01b0 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 >> v1:172.21.15.144:6789/0 conn(0x7f8dd40d01b0 legacy=0x7f8dd40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dbc003a00 0x7f8dbc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dbc004850 0x7f8dbc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc005200 0x7f8dbc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd415cec0 con 0x7f8dbc003a00 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc00c080 con 0x7f8dbc005200 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc00c240 con 0x7f8dbc004850 2023-11-28T11:52:13.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dbc003a00 0x7f8dbc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd37fe700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc005200 0x7f8dbc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd27fc700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dbc004850 0x7f8dbc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 -- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dbc004850 msgr2=0x7f8dbc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dbc004850 0x7f8dbc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 -- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc005200 msgr2=0x7f8dbc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc005200 0x7f8dbc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 -- 172.21.15.144:0/2422064227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0035b0 con 0x7f8dbc003a00 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd37fe700 1 -- 172.21.15.144:0/2422064227 reap_dead start 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dbc003a00 0x7f8dbc003e10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c240 tx=0x7f8dbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586dfd9c5893c60b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.927+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8007c10 con 0x7f8dbc003a00 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800cd80 con 0x7f8dbc003a00 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd1ffb700 1 -- 172.21.15.144:0/2422064227 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8005ba0 con 0x7f8dbc003a00 2023-11-28T11:52:13.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 -- 172.21.15.144:0/2422064227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dbc003a00 msgr2=0x7f8dbc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dbc003a00 0x7f8dbc003e10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c240 tx=0x7f8dbc005880 comp rx=0 tx=0).stop 2023-11-28T11:52:13.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 -- 172.21.15.144:0/2422064227 shutdown_connections 2023-11-28T11:52:13.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 --2- 172.21.15.144:0/2422064227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dbc003a00 0x7f8dbc003e10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 -- 172.21.15.144:0/2422064227 >> 172.21.15.144:0/2422064227 conn(0x7f8dd409c3f0 msgr2=0x7f8dd40d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:13.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 -- 172.21.15.144:0/2422064227 shutdown_connections 2023-11-28T11:52:13.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.928+0000 7f8dd9691700 1 -- 172.21.15.144:0/2422064227 wait complete. 2023-11-28T11:52:13.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.929+0000 7f8dd9691700 1 Processor -- start 2023-11-28T11:52:13.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.929+0000 7f8dd9691700 1 -- start start 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd9691700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc003a00 0x7f8dd40b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd9691700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dd4159540 0x7f8dd415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd9691700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 0x7f8dd40cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd9691700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4159cb0 con 0x7f8dd415f3f0 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd9691700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41599d0 con 0x7f8dbc003a00 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd9691700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd415f910 con 0x7f8dd4159540 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 0x7f8dd40cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 0x7f8dd40cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36466/0 (socket says 172.21.15.144:36466) 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd37fe700 1 -- 172.21.15.144:0/87884888 learned_addr learned my addr 172.21.15.144:0/87884888 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc003a00 0x7f8dd40b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.930+0000 7f8dd27fc700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dd4159540 0x7f8dd415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd37fe700 1 -- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dd4159540 msgr2=0x7f8dd415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd37fe700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dd4159540 0x7f8dd415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd37fe700 1 -- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc003a00 msgr2=0x7f8dd40b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd37fe700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc003a00 0x7f8dd40b8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd37fe700 1 -- 172.21.15.144:0/87884888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40afec0 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd37fe700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 0x7f8dd40cd660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40b01a0 tx=0x7f8dcc011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00e040 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00f040 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00f8f0 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd40afbe0 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.931+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40b2600 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.932+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd4068690 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.932+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8dcc00fa50 con 0x7f8dd415f3f0 2023-11-28T11:52:13.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.933+0000 7f8dbb7fe700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8dc00682f0 0x7f8dc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:13.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.933+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8dcc0862c0 con 0x7f8dd415f3f0 2023-11-28T11:52:13.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.933+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8dc00682f0 0x7f8dc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:13.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.933+0000 7f8dd2ffd700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8dc00682f0 0x7f8dc006a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8000c00 tx=0x7f8dc8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:13.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:13.937+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8dcc052b20 con 0x7f8dd415f3f0 2023-11-28T11:52:14.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.195+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f8dd40cf660 con 0x7f8dd415f3f0 2023-11-28T11:52:14.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.198+0000 7f8dbb7fe700 1 -- 172.21.15.144:0/87884888 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8dcc056140 con 0x7f8dd415f3f0 2023-11-28T11:52:14.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.200+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8dc00682f0 msgr2=0x7f8dc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8dc00682f0 0x7f8dc006a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8000c00 tx=0x7f8dc8007000 comp rx=0 tx=0).stop 2023-11-28T11:52:14.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 msgr2=0x7f8dd40cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 0x7f8dd40cd660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40b01a0 tx=0x7f8dcc011ed0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 shutdown_connections 2023-11-28T11:52:14.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dd415f3f0 0x7f8dd40cd660 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8dc00682f0 0x7f8dc006a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8dbc003a00 0x7f8dd40b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 --2- 172.21.15.144:0/87884888 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8dd4159540 0x7f8dd415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 >> 172.21.15.144:0/87884888 conn(0x7f8dd409c3f0 msgr2=0x7f8dd406c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:14.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 shutdown_connections 2023-11-28T11:52:14.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.201+0000 7f8dd9691700 1 -- 172.21.15.144:0/87884888 wait complete. 2023-11-28T11:52:14.204 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.0 2023-11-28T11:52:14.213 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:14.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.332+0000 7feaaabce700 1 Processor -- start 2023-11-28T11:52:14.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.332+0000 7feaaabce700 1 -- start start 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaaabce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa4140950 0x7feaa40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaaabce700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feaa40b2e70 0x7feaa406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaaabce700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa413ff00 con 0x7feaa40cd8c0 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaaabce700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa41404c0 con 0x7feaa40b2e70 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaaabce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa4140dc0 con 0x7feaa4140950 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaa916b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feaa40b2e70 0x7feaa406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaa896a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7feaa40cd8c0 0x7feaa40cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53182/0 (socket says 172.21.15.144:53182) 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.333+0000 7feaa896a700 1 -- 172.21.15.144:0/2932272023 learned_addr learned my addr 172.21.15.144:0/2932272023 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:14.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa3fff700 1 --2- 172.21.15.144:0/2932272023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa4140950 0x7feaa40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa916b700 1 -- 172.21.15.144:0/2932272023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa4140950 msgr2=0x7feaa40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa916b700 1 --2- 172.21.15.144:0/2932272023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa4140950 0x7feaa40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa916b700 1 -- 172.21.15.144:0/2932272023 >> v1:172.21.15.144:6789/0 conn(0x7feaa40cd8c0 legacy=0x7feaa40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa916b700 1 -- 172.21.15.144:0/2932272023 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa415aa80 con 0x7feaa40b2e70 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa37fe700 1 -- 172.21.15.144:0/2932272023 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236475650 0 0) 0x7feaa413ff00 con 0x7feaa40cd8c0 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa916b700 1 --2- 172.21.15.144:0/2932272023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40b2e70 0x7feaa406f850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009d90 tx=0x7fea9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631a9aa542d4d098 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa37fe700 1 -- 172.21.15.144:0/2932272023 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00e070 con 0x7feaa40b2e70 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa37fe700 1 -- 172.21.15.144:0/2932272023 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c00bbd0 con 0x7feaa40b2e70 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaa37fe700 1 -- 172.21.15.144:0/2932272023 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c012430 con 0x7feaa40b2e70 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaaabce700 1 -- 172.21.15.144:0/2932272023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40b2e70 msgr2=0x7feaa406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.334+0000 7feaaabce700 1 --2- 172.21.15.144:0/2932272023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40b2e70 0x7feaa406f850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009d90 tx=0x7fea9c00b330 comp rx=0 tx=0).stop 2023-11-28T11:52:14.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.335+0000 7feaaabce700 1 -- 172.21.15.144:0/2932272023 shutdown_connections 2023-11-28T11:52:14.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.335+0000 7feaaabce700 1 --2- 172.21.15.144:0/2932272023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40b2e70 0x7feaa406f850 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.335+0000 7feaaabce700 1 --2- 172.21.15.144:0/2932272023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa4140950 0x7feaa40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.335+0000 7feaaabce700 1 -- 172.21.15.144:0/2932272023 >> 172.21.15.144:0/2932272023 conn(0x7feaa406aed0 msgr2=0x7feaa406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:14.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.335+0000 7feaaabce700 1 -- 172.21.15.144:0/2932272023 shutdown_connections 2023-11-28T11:52:14.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.335+0000 7feaaabce700 1 -- 172.21.15.144:0/2932272023 wait complete. 2023-11-28T11:52:14.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.336+0000 7feaaabce700 1 Processor -- start 2023-11-28T11:52:14.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.336+0000 7feaaabce700 1 -- start start 2023-11-28T11:52:14.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaaabce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa40b2e70 0x7feaa40c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaaabce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 0x7feaa4142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaaabce700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa4140950 0x7feaa4140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaaabce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c52c0 con 0x7feaa40cd8c0 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaaabce700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c55a0 con 0x7feaa4140950 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaaabce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c5880 con 0x7feaa40b2e70 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 0x7feaa4142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 0x7feaa4142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36478/0 (socket says 172.21.15.144:36478) 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 -- 172.21.15.144:0/362419982 learned_addr learned my addr 172.21.15.144:0/362419982 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa896a700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa40b2e70 0x7feaa40c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:14.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 -- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa40b2e70 msgr2=0x7feaa40c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa916b700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa4140950 0x7feaa4140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa40b2e70 0x7feaa40c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 -- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa4140950 msgr2=0x7feaa4140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa4140950 0x7feaa4140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 -- 172.21.15.144:0/362419982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c009a70 con 0x7feaa40cd8c0 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa916b700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa4140950 0x7feaa4140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa3fff700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 0x7feaa4142b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fea9000d7d0 tx=0x7fea9000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea900125f0 con 0x7feaa40cd8c0 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.337+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea90012750 con 0x7feaa40cd8c0 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.338+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa40b35f0 con 0x7feaa40cd8c0 2023-11-28T11:52:14.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.338+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90012ac0 con 0x7feaa40cd8c0 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.338+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa40b3310 con 0x7feaa40cd8c0 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.339+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa4140150 con 0x7feaa40cd8c0 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.339+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea90012c20 con 0x7feaa40cd8c0 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.339+0000 7feaa17fa700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea880682f0 0x7fea8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.340+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea900861b0 con 0x7feaa40cd8c0 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.340+0000 7feaa896a700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea880682f0 0x7fea8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:14.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.340+0000 7feaa896a700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea880682f0 0x7fea8806a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea98001360 tx=0x7fea98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:14.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.344+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea90052a10 con 0x7feaa40cd8c0 2023-11-28T11:52:14.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.571+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaa40c5a40 con 0x7feaa40cd8c0 2023-11-28T11:52:14.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.571+0000 7feaa17fa700 1 -- 172.21.15.144:0/362419982 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fea90056030 con 0x7feaa40cd8c0 2023-11-28T11:52:14.575 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea880682f0 msgr2=0x7fea8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea880682f0 0x7fea8806a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fea98001360 tx=0x7fea98009040 comp rx=0 tx=0).stop 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 msgr2=0x7feaa4142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 0x7feaa4142b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fea9000d7d0 tx=0x7fea9000db90 comp rx=0 tx=0).stop 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 shutdown_connections 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fea880682f0 0x7fea8806a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.576 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa4140950 0x7feaa4140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.577 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa40b2e70 0x7feaa40c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.577 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 --2- 172.21.15.144:0/362419982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa40cd8c0 0x7feaa4142b80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:14.577 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 >> 172.21.15.144:0/362419982 conn(0x7feaa406aed0 msgr2=0x7feaa40cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:14.577 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.573+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 shutdown_connections 2023-11-28T11:52:14.577 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:14.574+0000 7feaaabce700 1 -- 172.21.15.144:0/362419982 wait complete. 2023-11-28T11:52:14.577 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:14.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:52:09.307915+0000', 'modified': '2023-11-28T11:52:12.326845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5055}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5055': {'gid': 5055, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6836/2826166019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 2826166019}, {'type': 'v1', 'addr': '172.21.15.144:6836', 'nonce': 2826166019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:52:14.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:52:14.990 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T11:52:15.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.114+0000 7f0499d87700 1 Processor -- start 2023-11-28T11:52:15.115 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.114+0000 7f0499d87700 1 -- start start 2023-11-28T11:52:15.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0499d87700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f04940d0a00 0x7f049415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0499d87700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f04940b2930 0x7f04940b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0499d87700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049406b420 con 0x7f04940ce040 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0499d87700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04940b32f0 con 0x7f04940b2930 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0499d87700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049406ff60 con 0x7f04940d0a00 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f04940b2930 0x7f04940b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f04940b2930 0x7f04940b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36488/0 (socket says 172.21.15.144:36488) 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 -- 172.21.15.144:0/4048450910 learned_addr learned my addr 172.21.15.144:0/4048450910 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0492ffd700 1 --2- 172.21.15.144:0/4048450910 >> v2:172.21.15.177:3300/0 conn(0x7f04940d0a00 0x7f049415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 -- 172.21.15.144:0/4048450910 >> v2:172.21.15.177:3300/0 conn(0x7f04940d0a00 msgr2=0x7f049415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 --2- 172.21.15.144:0/4048450910 >> v2:172.21.15.177:3300/0 conn(0x7f04940d0a00 0x7f049415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 -- 172.21.15.144:0/4048450910 >> v1:172.21.15.177:6789/0 conn(0x7f04940ce040 legacy=0x7f049406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 -- 172.21.15.144:0/4048450910 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049406b700 con 0x7f04940b2930 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f0493fff700 1 --2- 172.21.15.144:0/4048450910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940b2930 0x7f04940b2db0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f049406ed50 tx=0x7f048c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac361ba3f7cba73e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f04927fc700 1 -- 172.21.15.144:0/4048450910 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00e070 con 0x7f04940b2930 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f04927fc700 1 -- 172.21.15.144:0/4048450910 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c00b9c0 con 0x7f04940b2930 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.115+0000 7f04927fc700 1 -- 172.21.15.144:0/4048450910 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c00bd70 con 0x7f04940b2930 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 -- 172.21.15.144:0/4048450910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940b2930 msgr2=0x7f04940b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 --2- 172.21.15.144:0/4048450910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940b2930 0x7f04940b2db0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f049406ed50 tx=0x7f048c00b330 comp rx=0 tx=0).stop 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 -- 172.21.15.144:0/4048450910 shutdown_connections 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 --2- 172.21.15.144:0/4048450910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940b2930 0x7f04940b2db0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 --2- 172.21.15.144:0/4048450910 >> v2:172.21.15.177:3300/0 conn(0x7f04940d0a00 0x7f049415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 -- 172.21.15.144:0/4048450910 >> 172.21.15.144:0/4048450910 conn(0x7f049406aed0 msgr2=0x7f049406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 -- 172.21.15.144:0/4048450910 shutdown_connections 2023-11-28T11:52:15.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.116+0000 7f0499d87700 1 -- 172.21.15.144:0/4048450910 wait complete. 2023-11-28T11:52:15.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.117+0000 7f0499d87700 1 Processor -- start 2023-11-28T11:52:15.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.117+0000 7f0499d87700 1 -- start start 2023-11-28T11:52:15.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0499d87700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04940b2930 0x7f049414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0499d87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 0x7f04940b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0499d87700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04940d0a00 0x7f04940cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0499d87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940c5730 con 0x7f04940ce040 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0499d87700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940c4b40 con 0x7f04940b2930 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0499d87700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940c4e00 con 0x7f04940d0a00 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 0x7f04940b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 0x7f04940b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36490/0 (socket says 172.21.15.144:36490) 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 -- 172.21.15.144:0/2569427444 learned_addr learned my addr 172.21.15.144:0/2569427444 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f04937fe700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04940b2930 0x7f049414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0493fff700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04940d0a00 0x7f04940cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 -- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04940d0a00 msgr2=0x7f04940cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04940d0a00 0x7f04940cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 -- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04940b2930 msgr2=0x7f049414cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04940b2930 0x7f049414cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f0492ffd700 1 -- 172.21.15.144:0/2569427444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c009b30 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.118+0000 7f04937fe700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04940b2930 0x7f049414cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.119+0000 7f0492ffd700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 0x7f04940b3ee0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f048800d800 tx=0x7f048800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.119+0000 7f047bfff700 1 -- 172.21.15.144:0/2569427444 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04880125f0 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.119+0000 7f047bfff700 1 -- 172.21.15.144:0/2569427444 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488012750 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.119+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049406bcc0 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.119+0000 7f047bfff700 1 -- 172.21.15.144:0/2569427444 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488012b00 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.119+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049406b9e0 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.120+0000 7f047bfff700 1 -- 172.21.15.144:0/2569427444 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0488012c60 con 0x7f04940ce040 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.120+0000 7f047bfff700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04800682f0 0x7f048006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.121+0000 7f047bfff700 1 -- 172.21.15.144:0/2569427444 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0488086300 con 0x7f04940ce040 2023-11-28T11:52:15.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.121+0000 7f04937fe700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04800682f0 0x7f048006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.121+0000 7f04937fe700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04800682f0 0x7f048006a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f049415ea70 tx=0x7f0484009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.122+0000 7f04a5f10b80 1 Processor -- start 2023-11-28T11:52:15.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.122+0000 7f04a5f10b80 1 -- start start 2023-11-28T11:52:15.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c16a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c3e130 0x562252c40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c5c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252b00bc0 con 0x562252c40810 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252b6f350 con 0x562252c3e130 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x562252b68880 con 0x562252c41120 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c16a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c16a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36496/0 (socket says 172.21.15.144:36496) 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f04937fe700 1 -- 172.21.15.144:0/1341081120 learned_addr learned my addr 172.21.15.144:0/1341081120 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f0492ffd700 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c3e130 0x562252c40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.123+0000 7f0493fff700 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c5c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04937fe700 1 -- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 msgr2=0x562252c5c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04937fe700 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c5c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04937fe700 1 -- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c3e130 msgr2=0x562252c40be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04937fe700 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c3e130 0x562252c40be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04937fe700 1 -- 172.21.15.144:0/1341081120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622529ea5e0 con 0x562252c40810 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04937fe700 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c16a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f04840053f0 tx=0x7f0484011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f384290c0538cd64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04797fa700 1 -- 172.21.15.144:0/1341081120 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484003d70 con 0x562252c40810 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04797fa700 1 -- 172.21.15.144:0/1341081120 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0484021450 con 0x562252c40810 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.124+0000 7f04797fa700 1 -- 172.21.15.144:0/1341081120 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484021800 con 0x562252c40810 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1341081120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 msgr2=0x562252c16a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c16a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f04840053f0 tx=0x7f0484011000 comp rx=0 tx=0).stop 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1341081120 shutdown_connections 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c16a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c5c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1341081120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c3e130 0x562252c40be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1341081120 >> 172.21.15.144:0/1341081120 conn(0x562252c13d60 msgr2=0x562252c146f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1341081120 shutdown_connections 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1341081120 wait complete. 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 -- start start 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.125+0000 7f04a5f10b80 1 Processor -- start 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c3e130 0x562252c44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c41120 0x562252c53560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622529ec8d0 con 0x562252c40810 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252c449b0 con 0x562252c41120 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x562252c3f8d0 con 0x562252c3e130 2023-11-28T11:52:15.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f04a5f10b80 10 client.0 _ll_register_callbacks cb 0x562252a0e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f0492ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c53020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f0492ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c53020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36504/0 (socket says 172.21.15.144:36504) 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.126+0000 7f0492ffd700 1 -- 172.21.15.144:0/258439840 learned_addr learned my addr 172.21.15.144:0/258439840 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04937fe700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c3e130 0x562252c44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0493fff700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c41120 0x562252c53560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0492ffd700 1 -- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c3e130 msgr2=0x562252c44470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0492ffd700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c3e130 0x562252c44470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0492ffd700 1 -- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c41120 msgr2=0x562252c53560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0492ffd700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c41120 0x562252c53560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0492ffd700 1 -- 172.21.15.144:0/258439840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048400f040 con 0x562252c40810 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0493fff700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c41120 0x562252c53560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f0492ffd700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c53020 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0488056660 tx=0x7f0488086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d50ca67d0f37c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04727fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04727fc700 1 -- 172.21.15.144:0/258439840 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048805c2d0 con 0x562252c40810 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04727fc700 1 -- 172.21.15.144:0/258439840 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f048800fbd0 con 0x562252c40810 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562252b36140 con 0x562252c40810 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04727fc700 1 -- 172.21.15.144:0/258439840 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f048800fd90 con 0x562252c40810 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 10 client.5250 fetch_fsmap learned FSMap version 16 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x562252996d90 con 0x562252c40810 2023-11-28T11:52:15.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04727fc700 1 -- 172.21.15.144:0/258439840 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562252b36140 con 0x562252c40810 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04727fc700 1 -- 172.21.15.144:0/258439840 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f048805a120 con 0x562252c40810 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 10 client.5250 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 10 client.5250 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 20 client.5250 populate_metadata read hostname 'smithi144' 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c3f030 0x562252c78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 4 client.5250 mds_command: new command op to 5055 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.127+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 --> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622529ecf10 con 0x562252c3f030 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04937fe700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c3f030 0x562252c78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04727fc700 10 client.5250 ms_handle_connect on v2:172.21.15.144:6834/2826166019 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04937fe700 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c3f030 0x562252c78730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668ea8e825ad3c81 server_cookie=b1ba7e094e6a0ebe in_seq=0 out_seq=0 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04727fc700 1 -- 172.21.15.144:0/258439840 <== mds.0 v2:172.21.15.144:6834/2826166019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5622529ecf10 con 0x562252c3f030 2023-11-28T11:52:15.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04727fc700 10 client.5250 handle_command_reply: tid=0 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04a5f10b80 1 client.5250 shutdown 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 msgr2=0x562252c53020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.128+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c53020 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0488056660 tx=0x7f0488086050 comp rx=0 tx=0).stop 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 shutdown_connections 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c3f030 0x562252c78730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c41120 0x562252c53560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c3e130 0x562252c44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04727fc700 0 client.5250 ms_handle_reset on v2:172.21.15.144:6834/2826166019 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/258439840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c40810 0x562252c53020 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 >> 172.21.15.144:0/258439840 conn(0x562252c13d60 msgr2=0x562252c152d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 shutdown_connections 2023-11-28T11:52:15.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 1 -- 172.21.15.144:0/258439840 wait complete. 2023-11-28T11:52:15.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.129+0000 7f04a5f10b80 20 client.5250 trim_cache size 0 max 16384 2023-11-28T11:52:15.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.146+0000 7f04a5f10b80 1 Processor -- start 2023-11-28T11:52:15.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.146+0000 7f04a5f10b80 1 -- start start 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c10e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252c54770 con 0x562252c3e130 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252c2e510 con 0x562252c40810 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x562252c33c00 con 0x562252c41120 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c156e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36510/0 (socket says 172.21.15.144:36510) 2023-11-28T11:52:15.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 -- 172.21.15.144:0/853495057 learned_addr learned my addr 172.21.15.144:0/853495057 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f0492ffd700 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 -- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 msgr2=0x562252c10e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f0493fff700 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c10e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c10e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 -- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 msgr2=0x562252c336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 -- 172.21.15.144:0/853495057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562252b2f140 con 0x562252c3e130 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f0492ffd700 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f04937fe700 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c156e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x562252c120d0 tx=0x7f0484005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3f5b1e3ff30d19 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f0479ffb700 1 -- 172.21.15.144:0/853495057 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048402d320 con 0x562252c3e130 2023-11-28T11:52:15.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f0479ffb700 1 -- 172.21.15.144:0/853495057 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0484019030 con 0x562252c3e130 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.147+0000 7f0479ffb700 1 -- 172.21.15.144:0/853495057 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484021a30 con 0x562252c3e130 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 -- 172.21.15.144:0/853495057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 msgr2=0x562252c156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c156e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x562252c120d0 tx=0x7f0484005000 comp rx=0 tx=0).stop 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 -- 172.21.15.144:0/853495057 shutdown_connections 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c156e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c10e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/853495057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 -- 172.21.15.144:0/853495057 >> 172.21.15.144:0/853495057 conn(0x562252c13d60 msgr2=0x562252c7cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 -- 172.21.15.144:0/853495057 shutdown_connections 2023-11-28T11:52:15.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.148+0000 7f04a5f10b80 1 -- 172.21.15.144:0/853495057 wait complete. 2023-11-28T11:52:15.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.149+0000 7f04a5f10b80 1 -- start start 2023-11-28T11:52:15.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.149+0000 7f04a5f10b80 1 Processor -- start 2023-11-28T11:52:15.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c50fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c0f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252c124a0 con 0x562252c3e130 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562252c51a30 con 0x562252c40810 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x562252caa880 con 0x562252c41120 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04a5f10b80 10 client.0 _ll_register_callbacks cb 0x562252a0e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c50fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c50fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36516/0 (socket says 172.21.15.144:36516) 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 -- 172.21.15.144:0/1863858536 learned_addr learned my addr 172.21.15.144:0/1863858536 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f0493fff700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c0f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f0492ffd700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 -- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 msgr2=0x562252c0f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c0f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 -- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 msgr2=0x562252c440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f04937fe700 1 -- 172.21.15.144:0/1863858536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622529ea5e0 con 0x562252c3e130 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f0492ffd700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.150+0000 7f0493fff700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c0f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04937fe700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c50fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0484000fb0 tx=0x7f0484000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d02578ef5b9155 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04727fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04727fc700 1 -- 172.21.15.144:0/1863858536 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484021d00 con 0x562252c3e130 2023-11-28T11:52:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562252b26160 con 0x562252c3e130 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04727fc700 1 -- 172.21.15.144:0/1863858536 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f048402d7d0 con 0x562252c3e130 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04727fc700 1 -- 172.21.15.144:0/1863858536 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0484021e60 con 0x562252c3e130 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 10 client.5256 fetch_fsmap learned FSMap version 16 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5622529bd200 con 0x562252c3e130 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04727fc700 1 -- 172.21.15.144:0/1863858536 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562252b26160 con 0x562252c3e130 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04727fc700 1 -- 172.21.15.144:0/1863858536 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0484034070 con 0x562252c3e130 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 10 client.5256 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 10 client.5256 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 20 client.5256 populate_metadata read hostname 'smithi144' 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c53210 0x562252c0f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 4 client.5256 mds_command: new command op to 5055 tid=0[{"prefix": "status"}] 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 --> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562252b10170 con 0x562252c53210 2023-11-28T11:52:15.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.151+0000 7f0492ffd700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c53210 0x562252c0f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04727fc700 10 client.5256 ms_handle_connect on v2:172.21.15.144:6834/2826166019 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f0492ffd700 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c53210 0x562252c0f990 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fdc881f6f8d3a4 server_cookie=a5b75297b6441d77 in_seq=0 out_seq=0 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04727fc700 1 -- 172.21.15.144:0/1863858536 <== mds.0 v2:172.21.15.144:6834/2826166019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562252b10170 con 0x562252c53210 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04727fc700 10 client.5256 handle_command_reply: tid=0 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 client.5256 shutdown 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 msgr2=0x562252c50fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c50fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0484000fb0 tx=0x7f0484000f20 comp rx=0 tx=0).stop 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 shutdown_connections 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:6834/2826166019,v1:172.21.15.144:6836/2826166019] conn(0x562252c53210 0x562252c0f990 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562252c3e130 0x562252c50fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04727fc700 0 client.5256 ms_handle_reset on v2:172.21.15.144:6834/2826166019 2023-11-28T11:52:15.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x562252c41120 0x562252c0f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 --2- 172.21.15.144:0/1863858536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562252c40810 0x562252c440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 >> 172.21.15.144:0/1863858536 conn(0x562252c13d60 msgr2=0x562252c10e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 shutdown_connections 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 1 -- 172.21.15.144:0/1863858536 wait complete. 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f04a5f10b80 20 client.5256 trim_cache size 0 max 16384 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stdout: "cluster_fsid": "af726c13-c3f1-4279-9fca-12c6e4621b18", 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stdout: "whoami": 0, 2023-11-28T11:52:15.161 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 5055, 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "want_state": "up:active", 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "state": "up:active", 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs", 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "rank_uptime": 5.8420616120000002, 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "mdsmap_epoch": 16, 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch": 34, 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch_barrier": 34, 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout: "uptime": 13.189645495000001 2023-11-28T11:52:15.162 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:52:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04800682f0 msgr2=0x7f048006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f0499d87700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04800682f0 0x7f048006a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f049415ea70 tx=0x7f0484009040 comp rx=0 tx=0).stop 2023-11-28T11:52:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 msgr2=0x7f04940b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.152+0000 7f0499d87700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 0x7f04940b3ee0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f048800d800 tx=0x7f048800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 shutdown_connections 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f04800682f0 0x7f048006a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04940d0a00 0x7f04940cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04940b2930 0x7f049414cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 --2- 172.21.15.144:0/2569427444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04940ce040 0x7f04940b3ee0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 >> 172.21.15.144:0/2569427444 conn(0x7f049406aed0 msgr2=0x7f049415edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 shutdown_connections 2023-11-28T11:52:15.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.153+0000 7f0499d87700 1 -- 172.21.15.144:0/2569427444 wait complete. 2023-11-28T11:52:15.165 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:52:15.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 Processor -- start 2023-11-28T11:52:15.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 -- start start 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdefc071040 0x7fdefc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdefc06dee0 0x7fdefc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc071950 con 0x7fdefc06cdc0 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc161550 con 0x7fdefc06dee0 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.285+0000 7fdf02ce5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc15d060 con 0x7fdefc071040 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdefc06dee0 0x7fdefc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdefc06dee0 0x7fdefc06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36526/0 (socket says 172.21.15.144:36526) 2023-11-28T11:52:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 -- 172.21.15.144:0/2150898279 learned_addr learned my addr 172.21.15.144:0/2150898279 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdf00a81700 1 --2- 172.21.15.144:0/2150898279 >> v2:172.21.15.177:3300/0 conn(0x7fdefc071040 0x7fdefc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 -- 172.21.15.144:0/2150898279 >> v2:172.21.15.177:3300/0 conn(0x7fdefc071040 msgr2=0x7fdefc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 --2- 172.21.15.144:0/2150898279 >> v2:172.21.15.177:3300/0 conn(0x7fdefc071040 0x7fdefc071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 -- 172.21.15.144:0/2150898279 >> v1:172.21.15.177:6789/0 conn(0x7fdefc06cdc0 legacy=0x7fdefc06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 -- 172.21.15.144:0/2150898279 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc15d340 con 0x7fdefc06dee0 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefbfff700 1 --2- 172.21.15.144:0/2150898279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc06dee0 0x7fdefc06e330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0d0f50 tx=0x7fdee8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba425153c6d7931 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefb7fe700 1 -- 172.21.15.144:0/2150898279 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004230 con 0x7fdefc06dee0 2023-11-28T11:52:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefb7fe700 1 -- 172.21.15.144:0/2150898279 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee8004390 con 0x7fdefc06dee0 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.286+0000 7fdefb7fe700 1 -- 172.21.15.144:0/2150898279 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80036a0 con 0x7fdefc06dee0 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 -- 172.21.15.144:0/2150898279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc06dee0 msgr2=0x7fdefc06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/2150898279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc06dee0 0x7fdefc06e330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0d0f50 tx=0x7fdee8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 -- 172.21.15.144:0/2150898279 shutdown_connections 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/2150898279 >> v2:172.21.15.177:3300/0 conn(0x7fdefc071040 0x7fdefc071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/2150898279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc06dee0 0x7fdefc06e330 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 -- 172.21.15.144:0/2150898279 >> 172.21.15.144:0/2150898279 conn(0x7fdefc0d0000 msgr2=0x7fdefc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 -- 172.21.15.144:0/2150898279 shutdown_connections 2023-11-28T11:52:15.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.287+0000 7fdf02ce5700 1 -- 172.21.15.144:0/2150898279 wait complete. 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.288+0000 7fdf02ce5700 1 Processor -- start 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.288+0000 7fdf02ce5700 1 -- start start 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf02ce5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdefc06cdc0 0x7fdefc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf02ce5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdefc06dee0 0x7fdefc0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf02ce5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 0x7fdefc0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf02ce5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc159750 con 0x7fdefc071040 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf02ce5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0ae110 con 0x7fdefc06dee0 2023-11-28T11:52:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf02ce5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0ae3f0 con 0x7fdefc06cdc0 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 0x7fdefc0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 0x7fdefc0ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36534/0 (socket says 172.21.15.144:36534) 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 -- 172.21.15.144:0/3963656759 learned_addr learned my addr 172.21.15.144:0/3963656759 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdefbfff700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdefc06dee0 0x7fdefc0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 -- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdefc06cdc0 msgr2=0x7fdefc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdefc06cdc0 0x7fdefc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 -- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdefc06dee0 msgr2=0x7fdefc0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdefc06dee0 0x7fdefc0ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.289+0000 7fdf01282700 1 -- 172.21.15.144:0/3963656759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee8009550 con 0x7fdefc071040 2023-11-28T11:52:15.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.290+0000 7fdf01282700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 0x7fdefc0ada70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdef400b970 tx=0x7fdef400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.290+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef400c9b0 con 0x7fdefc071040 2023-11-28T11:52:15.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.290+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef400cb10 con 0x7fdefc071040 2023-11-28T11:52:15.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.290+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc15d900 con 0x7fdefc071040 2023-11-28T11:52:15.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.290+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4014440 con 0x7fdefc071040 2023-11-28T11:52:15.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.290+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc15d620 con 0x7fdefc071040 2023-11-28T11:52:15.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.291+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc06e870 con 0x7fdefc071040 2023-11-28T11:52:15.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.291+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdef401b030 con 0x7fdefc071040 2023-11-28T11:52:15.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.291+0000 7fdef97fa700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdee00682f0 0x7fdee006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:15.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.291+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdef4085860 con 0x7fdefc071040 2023-11-28T11:52:15.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.291+0000 7fdf00a81700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdee00682f0 0x7fdee006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:15.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.292+0000 7fdf00a81700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdee00682f0 0x7fdee006a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdef000a890 tx=0x7fdef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:15.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.296+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdef4052030 con 0x7fdefc071040 2023-11-28T11:52:15.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.522+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdefc0720a0 con 0x7fdefc071040 2023-11-28T11:52:15.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.522+0000 7fdef97fa700 1 -- 172.21.15.144:0/3963656759 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdef4019070 con 0x7fdefc071040 2023-11-28T11:52:15.523 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:52:15.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdee00682f0 msgr2=0x7fdee006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdee00682f0 0x7fdee006a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdef000a890 tx=0x7fdef0008040 comp rx=0 tx=0).stop 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 msgr2=0x7fdefc0ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 0x7fdefc0ada70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdef400b970 tx=0x7fdef400bd30 comp rx=0 tx=0).stop 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 shutdown_connections 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdefc071040 0x7fdefc0ada70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdee00682f0 0x7fdee006a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdefc06cdc0 0x7fdefc06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 --2- 172.21.15.144:0/3963656759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdefc06dee0 0x7fdefc0ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.524+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 >> 172.21.15.144:0/3963656759 conn(0x7fdefc0d0000 msgr2=0x7fdefc0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.525+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 shutdown_connections 2023-11-28T11:52:15.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:15.525+0000 7fdf02ce5700 1 -- 172.21.15.144:0/3963656759 wait complete. 2023-11-28T11:52:15.537 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:52:15.537 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:52:15.538 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:52:15.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.539 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:52:15.556 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1121:9a48:f2a:7a1a/64 scope link noprefixroute 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1a:fa:92:22:2e:a9 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:52:15.557 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::fccf:aaff:fe54:493c/64 scope link 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether ee:83:c8:ce:35:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec83:c8ff:fece:355d/64 scope link 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1a:fa:92:22:2e:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::18fa:92ff:fe22:2ea9/64 scope link 2023-11-28T11:52:15.558 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:52:15.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.559 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:52:15.622 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T11:52:15.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.623 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-11-28T11:52:15.682 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-11-28T11:52:15.683 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:52:15.683 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:52:15.683 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:52:15.683 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:52:15.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.683 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:52:15.683 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:52:15.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:52:15.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:52:15.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:52:15.684 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:52:15.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.788 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:52:15.804 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:52:15.805 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:52:15.805 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:52:15.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.805 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-11-28T11:52:15.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:52:15.872 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:52:15.928 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:52:15.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:52:15.929 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:52:15.985 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:52:15.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:15.986 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:52:16.072 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:52:16.073 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:52:16.073 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:52:16.081 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:52:16.081 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:52:16.081 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:52:16.081 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:52:16.081 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:52:16.117 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:52:16.120 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:52:16.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:52:16.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.153 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:52:16.235 INFO:teuthology.orchestra.run.smithi144.stdout:client5265 2023-11-28T11:52:16.236 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:16.236 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:16.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.293 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T11:52:16.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.356 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T11:52:16.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.415 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T11:52:16.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.474 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T11:52:16.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.537 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T11:52:16.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:16.594 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T11:52:16.656 INFO:teuthology.orchestra.run.smithi144.stdout:./ 2023-11-28T11:52:16.656 INFO:teuthology.orchestra.run.smithi144.stdout:./rfile 2023-11-28T11:52:16.656 INFO:teuthology.orchestra.run.smithi144.stdout:./parentdir 2023-11-28T11:52:16.656 INFO:teuthology.orchestra.run.smithi144.stdout:./parentdir/pfile 2023-11-28T11:52:16.656 INFO:teuthology.orchestra.run.smithi144.stdout:./parentdir/childdir 2023-11-28T11:52:16.656 INFO:teuthology.orchestra.run.smithi144.stdout:./parentdir/childdir/cfile 2023-11-28T11:52:16.657 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:52:16.657 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:52:16.657 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:52:16.657 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:52:16.657 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T11:52:16.657 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:52:16.763 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T11:52:16.955 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:52:16.955 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:52:16.955 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:52:16.956 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:52:16.956 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T11:52:16.956 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:52:17.062 INFO:teuthology.orchestra.run.smithi144.stdout:1099511627778 2023-11-28T11:52:17.252 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:52:17.252 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:52:17.252 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:52:17.252 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:52:17.252 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T11:52:17.252 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:52:17.361 INFO:teuthology.orchestra.run.smithi144.stdout:1099511627776 2023-11-28T11:52:17.551 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:52:17.551 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:52:17.551 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:52:17.551 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:52:17.552 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T11:52:17.552 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:52:17.664 INFO:teuthology.orchestra.run.smithi144.stdout:1099511628279 2023-11-28T11:52:17.854 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:52:17.854 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:52:17.854 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:52:17.854 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:52:17.854 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T11:52:17.854 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:52:17.959 INFO:teuthology.orchestra.run.smithi144.stdout:1099511627777 2023-11-28T11:52:18.149 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:52:18.149 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:52:18.149 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:52:18.149 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:52:18.149 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T11:52:18.149 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:52:18.256 INFO:teuthology.orchestra.run.smithi144.stdout:1099511628280 2023-11-28T11:52:18.445 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:18.445 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:18.505 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:18.505 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:18.563 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:18.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.723+0000 7fbc8fb5a700 1 Processor -- start 2023-11-28T11:52:18.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.723+0000 7fbc8fb5a700 1 -- start start 2023-11-28T11:52:18.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8fb5a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbc88158ce0 0x7fbc8815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:18.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8fb5a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc880ce4b0 0x7fbc8806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:18.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8fb5a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8806bf10 con 0x7fbc8806fb30 2023-11-28T11:52:18.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8fb5a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8815d730 con 0x7fbc880ce4b0 2023-11-28T11:52:18.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8fb5a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc881590b0 con 0x7fbc88158ce0 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8e0f7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc880ce4b0 0x7fbc8806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8d0f5700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbc8806fb30 0x7fbc880cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53254/0 (socket says 172.21.15.144:53254) 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8d0f5700 1 -- 172.21.15.144:0/3419705277 learned_addr learned my addr 172.21.15.144:0/3419705277 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8d8f6700 1 --2- 172.21.15.144:0/3419705277 >> v2:172.21.15.177:3300/0 conn(0x7fbc88158ce0 0x7fbc8815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/3419705277 >> v2:172.21.15.177:3300/0 conn(0x7fbc88158ce0 msgr2=0x7fbc8815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8e0f7700 1 --2- 172.21.15.144:0/3419705277 >> v2:172.21.15.177:3300/0 conn(0x7fbc88158ce0 0x7fbc8815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/3419705277 >> v1:172.21.15.144:6789/0 conn(0x7fbc8806fb30 legacy=0x7fbc880cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/3419705277 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880cb0d0 con 0x7fbc880ce4b0 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8c8f4700 1 -- 172.21.15.144:0/3419705277 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139660178 0 0) 0x7fbc8806bf10 con 0x7fbc8806fb30 2023-11-28T11:52:18.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8e0f7700 1 --2- 172.21.15.144:0/3419705277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880ce4b0 0x7fbc8806c3a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806fa30 tx=0x7fbc8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05a807f69ed7781 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8c8f4700 1 -- 172.21.15.144:0/3419705277 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8000e070 con 0x7fbc880ce4b0 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8c8f4700 1 -- 172.21.15.144:0/3419705277 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8000bbd0 con 0x7fbc880ce4b0 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.724+0000 7fbc8c8f4700 1 -- 172.21.15.144:0/3419705277 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80012430 con 0x7fbc880ce4b0 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/3419705277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880ce4b0 msgr2=0x7fbc8806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/3419705277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880ce4b0 0x7fbc8806c3a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbc8806fa30 tx=0x7fbc8000b330 comp rx=0 tx=0).stop 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/3419705277 shutdown_connections 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/3419705277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880ce4b0 0x7fbc8806c3a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/3419705277 >> v2:172.21.15.177:3300/0 conn(0x7fbc88158ce0 0x7fbc8815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/3419705277 >> 172.21.15.144:0/3419705277 conn(0x7fbc880cb670 msgr2=0x7fbc880cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/3419705277 shutdown_connections 2023-11-28T11:52:18.728 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.725+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/3419705277 wait complete. 2023-11-28T11:52:18.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.726+0000 7fbc8fb5a700 1 Processor -- start 2023-11-28T11:52:18.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.726+0000 7fbc8fb5a700 1 -- start start 2023-11-28T11:52:18.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8fb5a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc8806fb30 0x7fbc8806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:18.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8fb5a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc880ce4b0 0x7fbc8806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8fb5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 0x7fbc880c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8fb5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880af4f0 con 0x7fbc88158ce0 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8fb5a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b0330 con 0x7fbc8806fb30 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8fb5a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880af720 con 0x7fbc880ce4b0 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 0x7fbc880c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 0x7fbc880c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36552/0 (socket says 172.21.15.144:36552) 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/2948117916 learned_addr learned my addr 172.21.15.144:0/2948117916 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8d8f6700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc8806fb30 0x7fbc8806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8d0f5700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc880ce4b0 0x7fbc8806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc880ce4b0 msgr2=0x7fbc8806c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc880ce4b0 0x7fbc8806c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc8806fb30 msgr2=0x7fbc8806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc8806fb30 0x7fbc8806c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8e0f7700 1 -- 172.21.15.144:0/2948117916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80009a70 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.727+0000 7fbc8d8f6700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc8806fb30 0x7fbc8806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.728+0000 7fbc8e0f7700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 0x7fbc880c7c40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbc80013000 tx=0x7fbc80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.728+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80012430 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.728+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc80020070 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.728+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8806b8c0 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.728+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80004280 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.728+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc880cb3b0 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.729+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8806bba0 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.729+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc800043e0 con 0x7fbc88158ce0 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.729+0000 7fbc7e7fc700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc6c0682f0 0x7fbc6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:18.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.729+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc80022070 con 0x7fbc88158ce0 2023-11-28T11:52:18.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.729+0000 7fbc8d8f6700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc6c0682f0 0x7fbc6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:18.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.730+0000 7fbc8d8f6700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc6c0682f0 0x7fbc6c06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007a50 tx=0x7fbc84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:18.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.734+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbc800b95e0 con 0x7fbc88158ce0 2023-11-28T11:52:18.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.918+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc880745c0 con 0x7fbc88158ce0 2023-11-28T11:52:18.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.919+0000 7fbc7e7fc700 1 -- 172.21.15.144:0/2948117916 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fbc80054990 con 0x7fbc88158ce0 2023-11-28T11:52:18.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:18.920 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:10.310687+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:18.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc6c0682f0 msgr2=0x7fbc6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:18.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc6c0682f0 0x7fbc6c06a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007a50 tx=0x7fbc84008040 comp rx=0 tx=0).stop 2023-11-28T11:52:18.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 msgr2=0x7fbc880c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:18.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 0x7fbc880c7c40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbc80013000 tx=0x7fbc80004ad0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 shutdown_connections 2023-11-28T11:52:18.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc88158ce0 0x7fbc880c7c40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc6c0682f0 0x7fbc6c06a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc8806fb30 0x7fbc8806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 --2- 172.21.15.144:0/2948117916 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc880ce4b0 0x7fbc8806c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:18.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 >> 172.21.15.144:0/2948117916 conn(0x7fbc880cb670 msgr2=0x7fbc880719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:18.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 shutdown_connections 2023-11-28T11:52:18.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:18.921+0000 7fbc8fb5a700 1 -- 172.21.15.144:0/2948117916 wait complete. 2023-11-28T11:52:18.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:52:07.570239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:52:08.264687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:52:18.936 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:52:19.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.059+0000 7fc35c09d700 1 Processor -- start 2023-11-28T11:52:19.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.059+0000 7fc35c09d700 1 -- start start 2023-11-28T11:52:19.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35c09d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc354140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35c09d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc3540cdc90 0x7fc3540b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35c09d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35413ff00 con 0x7fc3540b2e70 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35c09d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3541404c0 con 0x7fc3540cdc90 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35c09d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415aa80 con 0x7fc3540cd8c0 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35a63a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc3540b2e70 0x7fc35406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53272/0 (socket says 172.21.15.144:53272) 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc35a63a700 1 -- 172.21.15.144:0/2207494254 learned_addr learned my addr 172.21.15.144:0/2207494254 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc359e39700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc354140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc359638700 1 --2- 172.21.15.144:0/2207494254 >> v2:172.21.15.177:3300/0 conn(0x7fc3540cdc90 0x7fc3540b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300500758 0 0) 0x7fc35413ff00 con 0x7fc3540b2e70 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3400035b0 con 0x7fc3540b2e70 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1277567153 0 0) 0x7fc3400035b0 con 0x7fc3540b2e70 2023-11-28T11:52:19.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 msgr2=0x7fc354140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc354140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 >> v2:172.21.15.177:3300/0 conn(0x7fc3540cdc90 msgr2=0x7fc3540b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 --2- 172.21.15.144:0/2207494254 >> v2:172.21.15.177:3300/0 conn(0x7fc3540cdc90 0x7fc3540b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540b3310 con 0x7fc3540b2e70 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fc34c0019f0 con 0x7fc3540b2e70 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 >> v1:172.21.15.144:6789/0 conn(0x7fc3540b2e70 legacy=0x7fc35406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340003a00 0x7fc340003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.060+0000 7fc358e37700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc340004430 0x7fc3400048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc340005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3541404c0 con 0x7fc340004430 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415aa80 con 0x7fc340003a00 2023-11-28T11:52:19.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34000ba10 con 0x7fc3540cd8c0 2023-11-28T11:52:19.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc340004430 0x7fc3400048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359e39700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340003a00 0x7fc340003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc35a63a700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc340005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 -- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 msgr2=0x7fc340005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:19.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc340005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 -- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340003a00 msgr2=0x7fc340003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:19.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340003a00 0x7fc340003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 -- 172.21.15.144:0/2207494254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400035b0 con 0x7fc340004430 2023-11-28T11:52:19.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc35a63a700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3540cd8c0 0x7fc340005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:19.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc359638700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc340004430 0x7fc3400048f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc344009b60 tx=0x7fc340005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8b8cf4e978df39 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400e6c0 con 0x7fc340004430 2023-11-28T11:52:19.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34400e820 con 0x7fc340004430 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.061+0000 7fc358e37700 1 -- 172.21.15.144:0/2207494254 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400eb90 con 0x7fc340004430 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35c09d700 1 -- 172.21.15.144:0/2207494254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc340004430 msgr2=0x7fc3400048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35c09d700 1 --2- 172.21.15.144:0/2207494254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc340004430 0x7fc3400048f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc344009b60 tx=0x7fc340005880 comp rx=0 tx=0).stop 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35a63a700 1 -- 172.21.15.144:0/2207494254 reap_dead start 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35c09d700 1 -- 172.21.15.144:0/2207494254 shutdown_connections 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35c09d700 1 -- 172.21.15.144:0/2207494254 >> 172.21.15.144:0/2207494254 conn(0x7fc35406aed0 msgr2=0x7fc35406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:19.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35c09d700 1 -- 172.21.15.144:0/2207494254 shutdown_connections 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.062+0000 7fc35c09d700 1 -- 172.21.15.144:0/2207494254 wait complete. 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.063+0000 7fc35c09d700 1 Processor -- start 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.063+0000 7fc35c09d700 1 -- start start 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35c09d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340004430 0x7fc35415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35c09d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 0x7fc3540cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35c09d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc35415cda0 0x7fc35415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35c09d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540cce70 con 0x7fc3540cc4c0 2023-11-28T11:52:19.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35c09d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540ce9a0 con 0x7fc340004430 2023-11-28T11:52:19.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35c09d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540cec80 con 0x7fc35415cda0 2023-11-28T11:52:19.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 0x7fc3540cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 0x7fc3540cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36572/0 (socket says 172.21.15.144:36572) 2023-11-28T11:52:19.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 -- 172.21.15.144:0/3132920971 learned_addr learned my addr 172.21.15.144:0/3132920971 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:19.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc35a63a700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc35415cda0 0x7fc35415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359e39700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340004430 0x7fc35415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 -- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc35415cda0 msgr2=0x7fc35415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc35415cda0 0x7fc35415d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 -- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340004430 msgr2=0x7fc35415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340004430 0x7fc35415b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 -- 172.21.15.144:0/3132920971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354140150 con 0x7fc3540cc4c0 2023-11-28T11:52:19.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359e39700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340004430 0x7fc35415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc359638700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 0x7fc3540cc930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc340005770 tx=0x7fc3440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.064+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc344021070 con 0x7fc3540cc4c0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.065+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc344013a00 con 0x7fc3540cc4c0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.065+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3540b35f0 con 0x7fc3540cc4c0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.065+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400e6c0 con 0x7fc3540cc4c0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.065+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3540ce100 con 0x7fc3540cc4c0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.065+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc354068690 con 0x7fc3540cc4c0 2023-11-28T11:52:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.066+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc344019030 con 0x7fc3540cc4c0 2023-11-28T11:52:19.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.066+0000 7fc34a7fc700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc338068320 0x7fc33806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.066+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc344086e90 con 0x7fc3540cc4c0 2023-11-28T11:52:19.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.066+0000 7fc359e39700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc338068320 0x7fc33806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.067+0000 7fc359e39700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc338068320 0x7fc33806a7d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc350009a60 tx=0x7fc350008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.071+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc3440536f0 con 0x7fc3540cc4c0 2023-11-28T11:52:19.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.250+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3540cfdb0 con 0x7fc3540cc4c0 2023-11-28T11:52:19.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.251+0000 7fc34a7fc700 1 -- 172.21.15.144:0/3132920971 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc344017020 con 0x7fc3540cc4c0 2023-11-28T11:52:19.252 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:52:19.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.252+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc338068320 msgr2=0x7fc33806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.252+0000 7fc35c09d700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc338068320 0x7fc33806a7d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc350009a60 tx=0x7fc350008040 comp rx=0 tx=0).stop 2023-11-28T11:52:19.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.252+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 msgr2=0x7fc3540cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.252+0000 7fc35c09d700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 0x7fc3540cc930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc340005770 tx=0x7fc3440047c0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 shutdown_connections 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc338068320 0x7fc33806a7d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc35415cda0 0x7fc35415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc340004430 0x7fc35415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 --2- 172.21.15.144:0/3132920971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3540cc4c0 0x7fc3540cc930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 >> 172.21.15.144:0/3132920971 conn(0x7fc35406aed0 msgr2=0x7fc354157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 shutdown_connections 2023-11-28T11:52:19.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.253+0000 7fc35c09d700 1 -- 172.21.15.144:0/3132920971 wait complete. 2023-11-28T11:52:19.266 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:52:19.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.388+0000 7fc061888700 1 Processor -- start 2023-11-28T11:52:19.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.388+0000 7fc061888700 1 -- start start 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.389+0000 7fc061888700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cd8c0 0x7fc05c140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.389+0000 7fc061888700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc05c06f850 0x7fc05c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.389+0000 7fc061888700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05c13ff00 con 0x7fc05c0cdc90 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.389+0000 7fc061888700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05c1404c0 con 0x7fc05c06f850 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.389+0000 7fc061888700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c15aa80 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc05c06f850 0x7fc05c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cd8c0 0x7fc05c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc05c06f850 0x7fc05c06fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:38854/0 (socket says 172.21.15.144:38854) 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cd8c0 0x7fc05c140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:48658/0 (socket says 172.21.15.144:48658) 2023-11-28T11:52:19.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 -- 172.21.15.144:0/855205023 learned_addr learned my addr 172.21.15.144:0/855205023 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05a7fc700 1 --1- 172.21.15.144:0/855205023 >> v1:172.21.15.177:6789/0 conn(0x7fc05c0cdc90 0x7fc05c0b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:53778/0 (socket says 172.21.15.144:53778) 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 -- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cd8c0 msgr2=0x7fc05c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 --2- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cd8c0 0x7fc05c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 -- 172.21.15.144:0/855205023 >> v1:172.21.15.177:6789/0 conn(0x7fc05c0cdc90 legacy=0x7fc05c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 -- 172.21.15.144:0/855205023 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0b3040 con 0x7fc05c06f850 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc05b7fe700 1 --2- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c06fcc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc054009e50 tx=0x7fc05400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf028f4c264a8f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc059ffb700 1 -- 172.21.15.144:0/855205023 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400e070 con 0x7fc05c06f850 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc059ffb700 1 -- 172.21.15.144:0/855205023 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05400b910 con 0x7fc05c06f850 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.390+0000 7fc059ffb700 1 -- 172.21.15.144:0/855205023 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400bcc0 con 0x7fc05c06f850 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 -- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 msgr2=0x7fc05c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 --2- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c06fcc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc054009e50 tx=0x7fc05400b330 comp rx=0 tx=0).stop 2023-11-28T11:52:19.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 -- 172.21.15.144:0/855205023 shutdown_connections 2023-11-28T11:52:19.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 --2- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c06fcc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 --2- 172.21.15.144:0/855205023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cd8c0 0x7fc05c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 -- 172.21.15.144:0/855205023 >> 172.21.15.144:0/855205023 conn(0x7fc05c06aed0 msgr2=0x7fc05c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:19.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 -- 172.21.15.144:0/855205023 shutdown_connections 2023-11-28T11:52:19.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.391+0000 7fc061888700 1 -- 172.21.15.144:0/855205023 wait complete. 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.392+0000 7fc061888700 1 Processor -- start 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.392+0000 7fc061888700 1 -- start start 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc061888700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc061888700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 0x7fc05c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc061888700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cdc90 0x7fc05c143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc061888700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0b5d60 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc061888700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0b57a0 con 0x7fc05c06f850 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc061888700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c0b5a80 con 0x7fc05c0cdc90 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 0x7fc05c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 0x7fc05c0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36586/0 (socket says 172.21.15.144:36586) 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 -- 172.21.15.144:0/3607578906 learned_addr learned my addr 172.21.15.144:0/3607578906 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05affd700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05b7fe700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cdc90 0x7fc05c143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 -- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cdc90 msgr2=0x7fc05c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cdc90 0x7fc05c143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 -- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 msgr2=0x7fc05c158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 -- 172.21.15.144:0/3607578906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc054009b30 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05b7fe700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cdc90 0x7fc05c143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.393+0000 7fc05a7fc700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 0x7fc05c0b4d50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00c4d0 tx=0x7fc04c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.394+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c005cb0 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.394+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c005e10 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.394+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c0b35f0 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.394+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0072d0 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.394+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c0b3310 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.395+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c140150 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.395+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc04c019030 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.395+0000 7fc043fff700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0480682f0 0x7fc04806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.396+0000 7fc05affd700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0480682f0 0x7fc04806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.396+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc04c084d90 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.396+0000 7fc05affd700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0480682f0 0x7fc04806a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc050009bc0 tx=0x7fc0500091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.400+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc04c04b5e0 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.582+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc05c068690 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.583+0000 7fc043fff700 1 -- 172.21.15.144:0/3607578906 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc04c017020 con 0x7fc05c0cd8c0 2023-11-28T11:52:19.584 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0480682f0 msgr2=0x7fc04806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0480682f0 0x7fc04806a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc050009bc0 tx=0x7fc0500091c0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 msgr2=0x7fc05c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 0x7fc05c0b4d50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00c4d0 tx=0x7fc04c00c7e0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 shutdown_connections 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0480682f0 0x7fc04806a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc05c0cdc90 0x7fc05c143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc05c06f850 0x7fc05c158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 --2- 172.21.15.144:0/3607578906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc05c0cd8c0 0x7fc05c0b4d50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 >> 172.21.15.144:0/3607578906 conn(0x7fc05c06aed0 msgr2=0x7fc05c13e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:19.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 shutdown_connections 2023-11-28T11:52:19.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.585+0000 7fc061888700 1 -- 172.21.15.144:0/3607578906 wait complete. 2023-11-28T11:52:19.598 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:52:19.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.720+0000 7f15c0c54700 1 Processor -- start 2023-11-28T11:52:19.722 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.721+0000 7f15c0c54700 1 -- start start 2023-11-28T11:52:19.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.721+0000 7f15c0c54700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc06dee0 0x7f15bc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.721+0000 7f15c0c54700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc15d060 con 0x7f15bc160540 2023-11-28T11:52:19.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.721+0000 7f15c0c54700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc15d340 con 0x7f15bc071040 2023-11-28T11:52:19.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.721+0000 7f15c0c54700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc15d620 con 0x7f15bc06dee0 2023-11-28T11:52:19.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15bad9d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f15bc160540 0x7f15bc1609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53306/0 (socket says 172.21.15.144:53306) 2023-11-28T11:52:19.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15bad9d700 1 -- 172.21.15.144:0/3647686550 learned_addr learned my addr 172.21.15.144:0/3647686550 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:19.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b9d9b700 1 --2- 172.21.15.144:0/3647686550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc06dee0 0x7f15bc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915860322 0 0) 0x7f15bc15d060 con 0x7f15bc160540 2023-11-28T11:52:19.734 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a40035b0 con 0x7f15bc160540 2023-11-28T11:52:19.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778220765 0 0) 0x7f15bc15d340 con 0x7f15bc071040 2023-11-28T11:52:19.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15bc15d060 con 0x7f15bc071040 2023-11-28T11:52:19.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b9d9b700 1 -- 172.21.15.144:0/3647686550 >> v1:172.21.15.177:6789/0 conn(0x7f15bc071040 legacy=0x7f15bc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613987904 0 0) 0x7f15a40035b0 con 0x7f15bc160540 2023-11-28T11:52:19.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b9d9b700 1 -- 172.21.15.144:0/3647686550 >> v1:172.21.15.144:6789/0 conn(0x7f15bc160540 legacy=0x7f15bc1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b9d9b700 1 -- 172.21.15.144:0/3647686550 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc15d900 con 0x7f15bc06dee0 2023-11-28T11:52:19.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b9d9b700 1 --2- 172.21.15.144:0/3647686550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc06dee0 0x7f15bc06e330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009870 tx=0x7f15ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a246a3710e22b8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40019f0 con 0x7f15bc06dee0 2023-11-28T11:52:19.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac004030 con 0x7f15bc06dee0 2023-11-28T11:52:19.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.722+0000 7f15b959a700 1 -- 172.21.15.144:0/3647686550 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0043a0 con 0x7f15bc06dee0 2023-11-28T11:52:19.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 -- 172.21.15.144:0/3647686550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc06dee0 msgr2=0x7f15bc06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 --2- 172.21.15.144:0/3647686550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc06dee0 0x7f15bc06e330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009870 tx=0x7f15ac009b80 comp rx=0 tx=0).stop 2023-11-28T11:52:19.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 -- 172.21.15.144:0/3647686550 shutdown_connections 2023-11-28T11:52:19.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 --2- 172.21.15.144:0/3647686550 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc06dee0 0x7f15bc06e330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 -- 172.21.15.144:0/3647686550 >> 172.21.15.144:0/3647686550 conn(0x7f15bc0d0000 msgr2=0x7f15bc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:19.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 -- 172.21.15.144:0/3647686550 shutdown_connections 2023-11-28T11:52:19.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.723+0000 7f15c0c54700 1 -- 172.21.15.144:0/3647686550 wait complete. 2023-11-28T11:52:19.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.725+0000 7f15c0c54700 1 Processor -- start 2023-11-28T11:52:19.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.725+0000 7f15c0c54700 1 -- start start 2023-11-28T11:52:19.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 0x7f15bc0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15bc071040 0x7f15bc155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc160540 0x7f15bc158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0c6500 con 0x7f15bc06dee0 2023-11-28T11:52:19.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc155ff0 con 0x7f15bc071040 2023-11-28T11:52:19.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0c5500 con 0x7f15bc160540 2023-11-28T11:52:19.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 0x7f15bc0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 0x7f15bc0c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36596/0 (socket says 172.21.15.144:36596) 2023-11-28T11:52:19.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 -- 172.21.15.144:0/987103342 learned_addr learned my addr 172.21.15.144:0/987103342 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:19.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 -- 172.21.15.144:0/987103342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc160540 msgr2=0x7f15bc158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc160540 0x7f15bc158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 -- 172.21.15.144:0/987103342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15bc071040 msgr2=0x7f15bc155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15bc071040 0x7f15bc155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 -- 172.21.15.144:0/987103342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0094a0 con 0x7f15bc06dee0 2023-11-28T11:52:19.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15ba59c700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 0x7f15bc0c5fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c500 tx=0x7f15b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b00052d0 con 0x7f15bc06dee0 2023-11-28T11:52:19.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0005450 con 0x7f15bc06dee0 2023-11-28T11:52:19.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc15ccd0 con 0x7f15bc06dee0 2023-11-28T11:52:19.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.726+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b00038c0 con 0x7f15bc06dee0 2023-11-28T11:52:19.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.727+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc06e870 con 0x7f15bc06dee0 2023-11-28T11:52:19.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.728+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc06a270 con 0x7f15bc06dee0 2023-11-28T11:52:19.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.728+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15b0019030 con 0x7f15bc06dee0 2023-11-28T11:52:19.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.728+0000 7f15aaffd700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f159c0682f0 0x7f159c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:19.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.728+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15b00850a0 con 0x7f15bc06dee0 2023-11-28T11:52:19.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.728+0000 7f15b9d9b700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f159c0682f0 0x7f159c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:19.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.729+0000 7f15b9d9b700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f159c0682f0 0x7f159c06a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0c6770 tx=0x7f15ac010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:19.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.733+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15b0051880 con 0x7f15bc06dee0 2023-11-28T11:52:19.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.913+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15bc0720a0 con 0x7f15bc06dee0 2023-11-28T11:52:19.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.914+0000 7f15aaffd700 1 -- 172.21.15.144:0/987103342 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15b0054ea0 con 0x7f15bc06dee0 2023-11-28T11:52:19.916 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f159c0682f0 msgr2=0x7f159c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f159c0682f0 0x7f159c06a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0c6770 tx=0x7f15ac010000 comp rx=0 tx=0).stop 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 msgr2=0x7f15bc0c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 0x7f15bc0c5fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c500 tx=0x7f15b000c8c0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 shutdown_connections 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15bc06dee0 0x7f15bc0c5fc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f159c0682f0 0x7f159c06a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f15bc160540 0x7f15bc158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 --2- 172.21.15.144:0/987103342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15bc071040 0x7f15bc155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:19.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 >> 172.21.15.144:0/987103342 conn(0x7f15bc0d0000 msgr2=0x7f15bc15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:19.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 shutdown_connections 2023-11-28T11:52:19.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:19.916+0000 7f15c0c54700 1 -- 172.21.15.144:0/987103342 wait complete. 2023-11-28T11:52:19.929 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:52:20.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.051+0000 7f744fcf0700 1 Processor -- start 2023-11-28T11:52:20.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.051+0000 7f744fcf0700 1 -- start start 2023-11-28T11:52:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744fcf0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f744806ba90 0x7f74480af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744fcf0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74480cbdf0 0x7f74480ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744fcf0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74480cc5f0 con 0x7f744815cef0 2023-11-28T11:52:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744fcf0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744815d470 con 0x7f744806ba90 2023-11-28T11:52:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744fcf0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744815d750 con 0x7f74480cbdf0 2023-11-28T11:52:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f744806ba90 0x7f74480af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744da8c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f744815cef0 0x7f744806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53316/0 (socket says 172.21.15.144:53316) 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f744806ba90 0x7f74480af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36606/0 (socket says 172.21.15.144:36606) 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744da8c700 1 -- 172.21.15.144:0/1524310030 learned_addr learned my addr 172.21.15.144:0/1524310030 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 -- 172.21.15.144:0/1524310030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74480cbdf0 msgr2=0x7f74480ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744e28d700 1 --2- 172.21.15.144:0/1524310030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74480cbdf0 0x7f74480ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 --2- 172.21.15.144:0/1524310030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74480cbdf0 0x7f74480ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 -- 172.21.15.144:0/1524310030 >> v1:172.21.15.144:6789/0 conn(0x7f744815cef0 legacy=0x7f744806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744ca8a700 1 -- 172.21.15.144:0/1524310030 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435139221 0 0) 0x7f74480cc5f0 con 0x7f744815cef0 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 -- 172.21.15.144:0/1524310030 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744815da30 con 0x7f744806ba90 2023-11-28T11:52:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744d28b700 1 --2- 172.21.15.144:0/1524310030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744806ba90 0x7f74480af400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7438009840 tx=0x7f7438009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdcf38f5e8196af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744ca8a700 1 -- 172.21.15.144:0/1524310030 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438011830 con 0x7f744806ba90 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744ca8a700 1 -- 172.21.15.144:0/1524310030 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7438011990 con 0x7f744806ba90 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.052+0000 7f744ca8a700 1 -- 172.21.15.144:0/1524310030 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438011d20 con 0x7f744806ba90 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 -- 172.21.15.144:0/1524310030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744806ba90 msgr2=0x7f74480af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 --2- 172.21.15.144:0/1524310030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744806ba90 0x7f74480af400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7438009840 tx=0x7f7438009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 -- 172.21.15.144:0/1524310030 shutdown_connections 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 --2- 172.21.15.144:0/1524310030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74480cbdf0 0x7f74480ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 --2- 172.21.15.144:0/1524310030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744806ba90 0x7f74480af400 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 -- 172.21.15.144:0/1524310030 >> 172.21.15.144:0/1524310030 conn(0x7f74480d0000 msgr2=0x7f74480d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 -- 172.21.15.144:0/1524310030 shutdown_connections 2023-11-28T11:52:20.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.053+0000 7f744fcf0700 1 -- 172.21.15.144:0/1524310030 wait complete. 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.054+0000 7f744fcf0700 1 Processor -- start 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.054+0000 7f744fcf0700 1 -- start start 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744fcf0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f744806ba90 0x7f74480aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744fcf0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74480cbdf0 0x7f74480c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744fcf0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 0x7f74481573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744fcf0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74481e42a0 con 0x7f744815cef0 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744fcf0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74481e3620 con 0x7f74480cbdf0 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744fcf0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74481e3900 con 0x7f744806ba90 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 0x7f74481573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 0x7f74481573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36614/0 (socket says 172.21.15.144:36614) 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 -- 172.21.15.144:0/3361336235 learned_addr learned my addr 172.21.15.144:0/3361336235 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:20.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744da8c700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f744806ba90 0x7f74480aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 -- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f744806ba90 msgr2=0x7f74480aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744d28b700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74480cbdf0 0x7f74480c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f744806ba90 0x7f74480aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 -- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74480cbdf0 msgr2=0x7f74480c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74480cbdf0 0x7f74480c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 -- 172.21.15.144:0/3361336235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380094a0 con 0x7f744815cef0 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.055+0000 7f744e28d700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 0x7f74481573d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f744000cd70 tx=0x7f744000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.056+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440003a20 con 0x7f744815cef0 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.056+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440003b80 con 0x7f744815cef0 2023-11-28T11:52:20.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.056+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744815dff0 con 0x7f744815cef0 2023-11-28T11:52:20.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.056+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440004020 con 0x7f744815cef0 2023-11-28T11:52:20.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.056+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744815dd10 con 0x7f744815cef0 2023-11-28T11:52:20.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.056+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74480cab90 con 0x7f744815cef0 2023-11-28T11:52:20.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.057+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f744001c020 con 0x7f744815cef0 2023-11-28T11:52:20.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.057+0000 7f743e7fc700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f742c0682f0 0x7f742c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.057+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7440012070 con 0x7f744815cef0 2023-11-28T11:52:20.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.057+0000 7f744da8c700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f742c0682f0 0x7f742c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.058+0000 7f744da8c700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f742c0682f0 0x7f742c06a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7444000f60 tx=0x7f7444009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.062+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7440052d20 con 0x7f744815cef0 2023-11-28T11:52:20.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.242+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f744804e620 con 0x7f744815cef0 2023-11-28T11:52:20.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.243+0000 7f743e7fc700 1 -- 172.21.15.144:0/3361336235 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7440056340 con 0x7f744815cef0 2023-11-28T11:52:20.244 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:52:20.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f742c0682f0 msgr2=0x7f742c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f742c0682f0 0x7f742c06a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7444000f60 tx=0x7f7444009040 comp rx=0 tx=0).stop 2023-11-28T11:52:20.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 msgr2=0x7f74481573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 0x7f74481573d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f744000cd70 tx=0x7f744000a570 comp rx=0 tx=0).stop 2023-11-28T11:52:20.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 shutdown_connections 2023-11-28T11:52:20.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f744815cef0 0x7f74481573d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f742c0682f0 0x7f742c06a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f744806ba90 0x7f74480aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 --2- 172.21.15.144:0/3361336235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74480cbdf0 0x7f74480c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 >> 172.21.15.144:0/3361336235 conn(0x7f74480d0000 msgr2=0x7f74480b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:20.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 shutdown_connections 2023-11-28T11:52:20.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.245+0000 7f744fcf0700 1 -- 172.21.15.144:0/3361336235 wait complete. 2023-11-28T11:52:20.257 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:52:20.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc0326e7700 1 Processor -- start 2023-11-28T11:52:20.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc0326e7700 1 -- start start 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc0326e7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc02c06f850 0x7fc02c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc0326e7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c13ff00 con 0x7fc02c140950 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc0326e7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c1404c0 con 0x7fc02c0cd8c0 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc0326e7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c140dc0 con 0x7fc02c06f850 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc030c84700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc02c06f850 0x7fc02c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.378+0000 7fc02b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc02c140950 0x7fc02c0b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53330/0 (socket says 172.21.15.144:53330) 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc02b7fe700 1 -- 172.21.15.144:0/1110830800 learned_addr learned my addr 172.21.15.144:0/1110830800 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc030c84700 1 -- 172.21.15.144:0/1110830800 >> v1:172.21.15.177:6789/0 conn(0x7fc02c0cd8c0 legacy=0x7fc02c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc030c84700 1 -- 172.21.15.144:0/1110830800 >> v1:172.21.15.144:6789/0 conn(0x7fc02c140950 legacy=0x7fc02c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc030c84700 1 -- 172.21.15.144:0/1110830800 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c15aa80 con 0x7fc02c06f850 2023-11-28T11:52:20.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc02affd700 1 -- 172.21.15.144:0/1110830800 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834582199 0 0) 0x7fc02c13ff00 con 0x7fc02c140950 2023-11-28T11:52:20.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc030c84700 1 --2- 172.21.15.144:0/1110830800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c06f850 0x7fc02c06fcf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc024009d90 tx=0x7fc02400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc47efeebc967d00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc02affd700 1 -- 172.21.15.144:0/1110830800 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400e070 con 0x7fc02c06f850 2023-11-28T11:52:20.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc02affd700 1 -- 172.21.15.144:0/1110830800 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02400bb90 con 0x7fc02c06f850 2023-11-28T11:52:20.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.379+0000 7fc02affd700 1 -- 172.21.15.144:0/1110830800 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024003a20 con 0x7fc02c06f850 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 -- 172.21.15.144:0/1110830800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c06f850 msgr2=0x7fc02c06fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 --2- 172.21.15.144:0/1110830800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c06f850 0x7fc02c06fcf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc024009d90 tx=0x7fc02400b330 comp rx=0 tx=0).stop 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 -- 172.21.15.144:0/1110830800 shutdown_connections 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 --2- 172.21.15.144:0/1110830800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c06f850 0x7fc02c06fcf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 -- 172.21.15.144:0/1110830800 >> 172.21.15.144:0/1110830800 conn(0x7fc02c06aed0 msgr2=0x7fc02c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 -- 172.21.15.144:0/1110830800 shutdown_connections 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.380+0000 7fc0326e7700 1 -- 172.21.15.144:0/1110830800 wait complete. 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.381+0000 7fc0326e7700 1 Processor -- start 2023-11-28T11:52:20.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.381+0000 7fc0326e7700 1 -- start start 2023-11-28T11:52:20.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc0326e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc02c06f850 0x7fc02c1f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc0326e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc02c0cd8c0 0x7fc02c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc0326e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 0x7fc02c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc0326e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0cb000 con 0x7fc02c140950 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc0326e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1f4450 con 0x7fc02c06f850 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc0326e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1f4120 con 0x7fc02c0cd8c0 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 0x7fc02c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 0x7fc02c0c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36628/0 (socket says 172.21.15.144:36628) 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 -- 172.21.15.144:0/33463453 learned_addr learned my addr 172.21.15.144:0/33463453 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc02b7fe700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc02c0cd8c0 0x7fc02c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 -- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc02c0cd8c0 msgr2=0x7fc02c0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc02bfff700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc02c06f850 0x7fc02c1f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc02c0cd8c0 0x7fc02c0c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 -- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc02c06f850 msgr2=0x7fc02c1f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc02c06f850 0x7fc02c1f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 -- 172.21.15.144:0/33463453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc024009a70 con 0x7fc02c140950 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc02bfff700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc02c06f850 0x7fc02c1f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.382+0000 7fc030c84700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 0x7fc02c0c8df0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc02400b730 tx=0x7fc024004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.383+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024012840 con 0x7fc02c140950 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.383+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0240129a0 con 0x7fc02c140950 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.383+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024012d50 con 0x7fc02c140950 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.383+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0b3310 con 0x7fc02c140950 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.383+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c0b3040 con 0x7fc02c140950 2023-11-28T11:52:20.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.384+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0b35f0 con 0x7fc02c140950 2023-11-28T11:52:20.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.384+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc02402c070 con 0x7fc02c140950 2023-11-28T11:52:20.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.384+0000 7fc028ff9700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0180682f0 0x7fc01806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.385+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc024087e90 con 0x7fc02c140950 2023-11-28T11:52:20.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.385+0000 7fc02bfff700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0180682f0 0x7fc01806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.385+0000 7fc02bfff700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0180682f0 0x7fc01806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc02c13f900 tx=0x7fc01c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.389+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc0240546f0 con 0x7fc02c140950 2023-11-28T11:52:20.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.588+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc02c0ce1c0 con 0x7fc02c140950 2023-11-28T11:52:20.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.588+0000 7fc028ff9700 1 -- 172.21.15.144:0/33463453 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc02402a080 con 0x7fc02c140950 2023-11-28T11:52:20.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0180682f0 msgr2=0x7fc01806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0180682f0 0x7fc01806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc02c13f900 tx=0x7fc01c009040 comp rx=0 tx=0).stop 2023-11-28T11:52:20.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 msgr2=0x7fc02c0c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 0x7fc02c0c8df0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc02400b730 tx=0x7fc024004ad0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 shutdown_connections 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc02c140950 0x7fc02c0c8df0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0180682f0 0x7fc01806a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc02c06f850 0x7fc02c1f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 --2- 172.21.15.144:0/33463453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc02c0cd8c0 0x7fc02c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 >> 172.21.15.144:0/33463453 conn(0x7fc02c06aed0 msgr2=0x7fc02c0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 shutdown_connections 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.590+0000 7fc0326e7700 1 -- 172.21.15.144:0/33463453 wait complete. 2023-11-28T11:52:20.592 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:52:20.603 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:52:20.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:20.603 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:52:20.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:52:20.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:52:20.673 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:52:20.695 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:20.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.854+0000 7f1f56109700 1 Processor -- start 2023-11-28T11:52:20.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.854+0000 7f1f56109700 1 -- start start 2023-11-28T11:52:20.857 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f56109700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f5015ca50 0x7f1f5006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.857 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f56109700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f56109700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f500cf5e0 con 0x7f1f5006b8c0 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f56109700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f500cd290 con 0x7f1f5015ca50 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f56109700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5006bd60 con 0x7f1f500cce10 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4f7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f5015ca50 0x7f1f5006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1f5006b8c0 0x7f1f500cc700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53344/0 (socket says 172.21.15.144:53344) 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4effd700 1 -- 172.21.15.144:0/1812189812 learned_addr learned my addr 172.21.15.144:0/1812189812 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4ffff700 1 --2- 172.21.15.144:0/1812189812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4f7fe700 1 -- 172.21.15.144:0/1812189812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 msgr2=0x7f1f500cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4f7fe700 1 --2- 172.21.15.144:0/1812189812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4f7fe700 1 -- 172.21.15.144:0/1812189812 >> v1:172.21.15.144:6789/0 conn(0x7f1f5006b8c0 legacy=0x7f1f500cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.858 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4f7fe700 1 -- 172.21.15.144:0/1812189812 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500cecb0 con 0x7f1f5015ca50 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.855+0000 7f1f4e7fc700 1 -- 172.21.15.144:0/1812189812 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848873978 0 0) 0x7f1f500cf5e0 con 0x7f1f5006b8c0 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f4f7fe700 1 --2- 172.21.15.144:0/1812189812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f5006b380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009840 tx=0x7f1f38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9231ac09ad99bf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f4e7fc700 1 -- 172.21.15.144:0/1812189812 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38011830 con 0x7f1f5015ca50 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f4e7fc700 1 -- 172.21.15.144:0/1812189812 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f38011990 con 0x7f1f5015ca50 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f4e7fc700 1 -- 172.21.15.144:0/1812189812 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f38011d20 con 0x7f1f5015ca50 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 -- 172.21.15.144:0/1812189812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 msgr2=0x7f1f5006b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 --2- 172.21.15.144:0/1812189812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f5006b380 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009840 tx=0x7f1f38009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 -- 172.21.15.144:0/1812189812 shutdown_connections 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 --2- 172.21.15.144:0/1812189812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 --2- 172.21.15.144:0/1812189812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f5006b380 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 -- 172.21.15.144:0/1812189812 >> 172.21.15.144:0/1812189812 conn(0x7f1f5009a230 msgr2=0x7f1f5008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 -- 172.21.15.144:0/1812189812 shutdown_connections 2023-11-28T11:52:20.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.856+0000 7f1f56109700 1 -- 172.21.15.144:0/1812189812 wait complete. 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.857+0000 7f1f56109700 1 Processor -- start 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.858+0000 7f1f56109700 1 -- start start 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.858+0000 7f1f56109700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f501589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500b6ed0 con 0x7f1f5015ca50 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500b5c20 con 0x7f1f5006b8c0 2023-11-28T11:52:20.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500b6bf0 con 0x7f1f500cce10 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f501589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f501589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36640/0 (socket says 172.21.15.144:36640) 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 -- 172.21.15.144:0/155065328 learned_addr learned my addr 172.21.15.144:0/155065328 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4effd700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4f7fe700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 -- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 msgr2=0x7f1f500b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 -- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 msgr2=0x7f1f500b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 -- 172.21.15.144:0/155065328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380094a0 con 0x7f1f5015ca50 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f4ffff700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f501589c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1f48007e40 tx=0x7f1f4800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800ed30 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f48014420 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f500cf270 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f480147d0 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.859+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f500cef90 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.860+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5015ce20 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.861+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f48012050 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.861+0000 7f1f47fff700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f300682f0 0x7f1f3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.861+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f48086210 con 0x7f1f5015ca50 2023-11-28T11:52:20.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.861+0000 7f1f4f7fe700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f300682f0 0x7f1f3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:20.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.862+0000 7f1f4f7fe700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f300682f0 0x7f1f3006a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f380090b0 tx=0x7f1f3801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:20.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:20.865+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f48052a70 con 0x7f1f5015ca50 2023-11-28T11:52:21.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.091+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f500707d0 con 0x7f1f5015ca50 2023-11-28T11:52:21.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.091+0000 7f1f47fff700 1 -- 172.21.15.144:0/155065328 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f1f48056090 con 0x7f1f5015ca50 2023-11-28T11:52:21.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:21.092 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:21.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f300682f0 msgr2=0x7f1f3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f300682f0 0x7f1f3006a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f380090b0 tx=0x7f1f3801c040 comp rx=0 tx=0).stop 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 msgr2=0x7f1f501589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f501589c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1f48007e40 tx=0x7f1f4800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 shutdown_connections 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015ca50 0x7f1f501589c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f300682f0 0x7f1f3006a7a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 --2- 172.21.15.144:0/155065328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cce10 0x7f1f500b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 >> 172.21.15.144:0/155065328 conn(0x7f1f5009a230 msgr2=0x7f1f500d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 shutdown_connections 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.093+0000 7f1f56109700 1 -- 172.21.15.144:0/155065328 wait complete. 2023-11-28T11:52:21.095 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:21.107 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T11:52:21.257 INFO:teuthology.orchestra.run.smithi144.stdout:{"message":"","return_code":0} 2023-11-28T11:52:21.258 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:52:21.258 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:21.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.377+0000 7f38c61a0700 1 Processor -- start 2023-11-28T11:52:21.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.378+0000 7f38c61a0700 1 -- start start 2023-11-28T11:52:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.378+0000 7f38c61a0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 0x7f38c006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.378+0000 7f38c61a0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c006d170 con 0x7f38c006a980 2023-11-28T11:52:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.378+0000 7f38c61a0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c015dd20 con 0x7f38c00b4960 2023-11-28T11:52:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.378+0000 7f38c61a0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c015d900 con 0x7f38c00af0c0 2023-11-28T11:52:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38c599f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f38c006a980 0x7f38c00b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53358/0 (socket says 172.21.15.144:53358) 2023-11-28T11:52:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38c599f700 1 -- 172.21.15.144:0/914231607 learned_addr learned my addr 172.21.15.144:0/914231607 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38c519e700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 0x7f38c006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059849895 0 0) 0x7f38c006d170 con 0x7f38c006a980 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38ac0035b0 con 0x7f38c006a980 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616058051 0 0) 0x7f38c015dd20 con 0x7f38c00b4960 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c006d170 con 0x7f38c00b4960 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282132553 0 0) 0x7f38ac0035b0 con 0x7f38c006a980 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 msgr2=0x7f38c006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 0x7f38c006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 >> v1:172.21.15.177:6789/0 conn(0x7f38c00b4960 legacy=0x7f38c006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c006d3a0 con 0x7f38c006a980 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f38b80019f0 con 0x7f38c006a980 2023-11-28T11:52:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 >> v1:172.21.15.144:6789/0 conn(0x7f38c006a980 legacy=0x7f38c00b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 0x7f38ac003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38ac004120 0x7f38ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ac004d90 0x7f38ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c015d900 con 0x7f38ac004d90 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac00bba0 con 0x7f38ac004120 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac00bd30 con 0x7f38c00af0c0 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38c599f700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ac004d90 0x7f38ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38c499d700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38ac004120 0x7f38ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.379+0000 7f38c519e700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 0x7f38ac003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 -- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 msgr2=0x7f38ac003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c00af0c0 0x7f38ac003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 -- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38ac004120 msgr2=0x7f38ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38ac004120 0x7f38ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 -- 172.21.15.144:0/914231607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac0035b0 con 0x7f38ac004d90 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c499d700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38ac004120 0x7f38ac004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ac004d90 0x7f38ac005200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f38b800a7c0 tx=0x7f38ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9558ca8f9b5460b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b80072d0 con 0x7f38ac004d90 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b8007450 con 0x7f38ac004d90 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38b7fff700 1 -- 172.21.15.144:0/914231607 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8007800 con 0x7f38ac004d90 2023-11-28T11:52:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c61a0700 1 -- 172.21.15.144:0/914231607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ac004d90 msgr2=0x7f38ac005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c61a0700 1 --2- 172.21.15.144:0/914231607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ac004d90 0x7f38ac005200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f38b800a7c0 tx=0x7f38ac005880 comp rx=0 tx=0).stop 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c599f700 1 -- 172.21.15.144:0/914231607 reap_dead start 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c61a0700 1 -- 172.21.15.144:0/914231607 shutdown_connections 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c61a0700 1 -- 172.21.15.144:0/914231607 >> 172.21.15.144:0/914231607 conn(0x7f38c006b380 msgr2=0x7f38c006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c61a0700 1 -- 172.21.15.144:0/914231607 shutdown_connections 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.380+0000 7f38c61a0700 1 -- 172.21.15.144:0/914231607 wait complete. 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 Processor -- start 2023-11-28T11:52:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 -- start start 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38c015bed0 0x7f38c014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c0151560 0x7f38c00b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 0x7f38c015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b0e90 con 0x7f38c00b0a20 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c014fa80 con 0x7f38c015bed0 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c61a0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c014fd60 con 0x7f38c0151560 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c599f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 0x7f38c015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c519e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38c015bed0 0x7f38c014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c599f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 0x7f38c015efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36658/0 (socket says 172.21.15.144:36658) 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c599f700 1 -- 172.21.15.144:0/619775841 learned_addr learned my addr 172.21.15.144:0/619775841 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.382+0000 7f38c499d700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c0151560 0x7f38c00b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c599f700 1 -- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c0151560 msgr2=0x7f38c00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c599f700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c0151560 0x7f38c00b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c599f700 1 -- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38c015bed0 msgr2=0x7f38c014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c599f700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38c015bed0 0x7f38c014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c599f700 1 -- 172.21.15.144:0/619775841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c015d240 con 0x7f38c00b0a20 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c599f700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 0x7f38c015efb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f38b8001c30 tx=0x7f38b8001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b80019b0 con 0x7f38c00b0a20 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b80072e0 con 0x7f38c00b0a20 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c015cf60 con 0x7f38c00b0a20 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8012640 con 0x7f38c00b0a20 2023-11-28T11:52:21.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.383+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c015d520 con 0x7f38c00b0a20 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.384+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c0061e80 con 0x7f38c00b0a20 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.385+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38b8010070 con 0x7f38c00b0a20 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.385+0000 7f38b5ffb700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38a40682f0 0x7f38a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.385+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38b8086e80 con 0x7f38c00b0a20 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.385+0000 7f38c519e700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38a40682f0 0x7f38a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.385+0000 7f38c519e700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38a40682f0 0x7f38a406a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009c60 tx=0x7f38b0009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:21.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.389+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f38b80537a0 con 0x7f38c00b0a20 2023-11-28T11:52:21.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.615+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38c006b160 con 0x7f38c00b0a20 2023-11-28T11:52:21.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.615+0000 7f38b5ffb700 1 -- 172.21.15.144:0/619775841 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f38b8018090 con 0x7f38c00b0a20 2023-11-28T11:52:21.617 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:21.617 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:21.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38a40682f0 msgr2=0x7f38a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38a40682f0 0x7f38a406a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009c60 tx=0x7f38b0009210 comp rx=0 tx=0).stop 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 msgr2=0x7f38c015efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 0x7f38c015efb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f38b8001c30 tx=0x7f38b8001d00 comp rx=0 tx=0).stop 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 shutdown_connections 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c00b0a20 0x7f38c015efb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38a40682f0 0x7f38a406a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38c015bed0 0x7f38c014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.617+0000 7f38c61a0700 1 --2- 172.21.15.144:0/619775841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38c0151560 0x7f38c00b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:21.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.618+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 >> 172.21.15.144:0/619775841 conn(0x7f38c006b380 msgr2=0x7f38c00b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:21.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.618+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 shutdown_connections 2023-11-28T11:52:21.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:21.618+0000 7f38c61a0700 1 -- 172.21.15.144:0/619775841 wait complete. 2023-11-28T11:52:21.620 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:21.631 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2023-11-28T11:52:21.771 INFO:teuthology.orchestra.run.smithi144.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T11:52:21.772 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T11:52:22.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:52:22.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:52:31.773 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T11:52:31.824 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.824+0000 7fa625c99080 1 Processor -- start 2023-11-28T11:52:31.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.824+0000 7fa625c99080 1 -- start start 2023-11-28T11:52:31.827 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.824+0000 7fa625c99080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac49da9d0 0x55cac4a68820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.827 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa625c99080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac4a68450 0x55cac49be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa625c99080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 0x55cac4a6ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa625c99080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4938000 con 0x55cac4a6cde0 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa625c99080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac48cafe0 con 0x55cac4a68450 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa625c99080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cac4a67a30 con 0x55cac49da9d0 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 0x55cac4a6ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 0x55cac4a6ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36670/0 (socket says 172.21.15.144:36670) 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 -- 172.21.15.144:0/4187839594 learned_addr learned my addr 172.21.15.144:0/4187839594 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61b59b700 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac4a68450 0x55cac49be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61bd9c700 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac49da9d0 0x55cac4a68820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 -- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac49da9d0 msgr2=0x55cac4a68820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac49da9d0 0x55cac4a68820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 -- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac4a68450 msgr2=0x55cac49be5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac4a68450 0x55cac49be5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 -- 172.21.15.144:0/4187839594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610005d20 con 0x55cac4a6cde0 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61bd9c700 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac49da9d0 0x55cac4a68820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61c59d700 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 0x55cac4a6ac40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa6140015c0 tx=0x7fa6140018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd605fd24eed409 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61a599700 1 -- 172.21.15.144:0/4187839594 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140062e0 con 0x55cac4a6cde0 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61a599700 1 -- 172.21.15.144:0/4187839594 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa614003450 con 0x55cac4a6cde0 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.825+0000 7fa61a599700 1 -- 172.21.15.144:0/4187839594 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614003800 con 0x55cac4a6cde0 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 -- 172.21.15.144:0/4187839594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 msgr2=0x55cac4a6ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 0x55cac4a6ac40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa6140015c0 tx=0x7fa6140018d0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 -- 172.21.15.144:0/4187839594 shutdown_connections 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a6cde0 0x55cac4a6ac40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac49da9d0 0x55cac4a68820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 --2- 172.21.15.144:0/4187839594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac4a68450 0x55cac49be5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 -- 172.21.15.144:0/4187839594 >> 172.21.15.144:0/4187839594 conn(0x55cac49bcb60 msgr2=0x55cac498f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:31.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 -- 172.21.15.144:0/4187839594 shutdown_connections 2023-11-28T11:52:31.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.826+0000 7fa625c99080 1 -- 172.21.15.144:0/4187839594 wait complete. 2023-11-28T11:52:31.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 Processor -- start 2023-11-28T11:52:31.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 -- start start 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac49da9d0 0x55cac4a61680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a68450 0x55cac4a6abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac4a6cde0 0x55cac4a6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4a64cd0 con 0x55cac4a68450 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac49becb0 con 0x55cac49da9d0 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa625c99080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cac49be9d0 con 0x55cac4a6cde0 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.827+0000 7fa61b59b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a68450 0x55cac4a6abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a68450 0x55cac4a6abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36674/0 (socket says 172.21.15.144:36674) 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 -- 172.21.15.144:0/3609156727 learned_addr learned my addr 172.21.15.144:0/3609156727 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61bd9c700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac49da9d0 0x55cac4a61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61c59d700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac4a6cde0 0x55cac4a6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 -- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac4a6cde0 msgr2=0x55cac4a6b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac4a6cde0 0x55cac4a6b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 -- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac49da9d0 msgr2=0x55cac4a61680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cac49da9d0 0x55cac4a61680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 -- 172.21.15.144:0/3609156727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa614006d90 con 0x55cac4a68450 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61c59d700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55cac4a6cde0 0x55cac4a6b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa61b59b700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a68450 0x55cac4a6abb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa60c002d30 tx=0x7fa60c00c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa619597700 1 -- 172.21.15.144:0/3609156727 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c016030 con 0x55cac4a68450 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa619597700 1 -- 172.21.15.144:0/3609156727 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c008070 con 0x55cac4a68450 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cac48d6830 con 0x55cac4a68450 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa619597700 1 -- 172.21.15.144:0/3609156727 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00cd60 con 0x55cac4a68450 2023-11-28T11:52:31.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.828+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cac48e3360 con 0x55cac4a68450 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.830+0000 7fa619597700 1 -- 172.21.15.144:0/3609156727 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa60c0019d0 con 0x55cac4a68450 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.830+0000 7fa619597700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa608066c40 0x7fa6080690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.830+0000 7fa619597700 1 -- 172.21.15.144:0/3609156727 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa60c017070 con 0x55cac4a68450 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.830+0000 7fa61bd9c700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa608066c40 0x7fa6080690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.830+0000 7fa625c99080 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x55cac4a66c10 0x55cac4a62900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.830+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 --> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55cac4a660e0 con 0x55cac4a66c10 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.831+0000 7fa61c59d700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x55cac4a66c10 0x55cac4a62900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.831+0000 7fa61bd9c700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa608066c40 0x7fa6080690f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa610015040 tx=0x7fa61000f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.831+0000 7fa61c59d700 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x55cac4a66c10 0x55cac4a62900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa61c59d700 1 -- 172.21.15.144:0/3609156727 <== osd.7 v2:172.21.15.177:6816/103773 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa6140035c0 con 0x55cac4a66c10 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x55cac4a66c10 msgr2=0x55cac4a62900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x55cac4a66c10 0x55cac4a62900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa608066c40 msgr2=0x7fa6080690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa608066c40 0x7fa6080690f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa610015040 tx=0x7fa61000f000 comp rx=0 tx=0).stop 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a68450 msgr2=0x55cac4a6abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 --2- 172.21.15.144:0/3609156727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cac4a68450 0x55cac4a6abb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa60c002d30 tx=0x7fa60c00c530 comp rx=0 tx=0).stop 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa61c59d700 1 -- 172.21.15.144:0/3609156727 reap_dead start 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 shutdown_connections 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 >> 172.21.15.144:0/3609156727 conn(0x55cac49bcb60 msgr2=0x55cac498f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 shutdown_connections 2023-11-28T11:52:31.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.832+0000 7fa625c99080 1 -- 172.21.15.144:0/3609156727 wait complete. 2023-11-28T11:52:31.838 INFO:teuthology.orchestra.run.smithi144.stdout:mytag 2023-11-28T11:52:31.838 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T11:52:31.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 Processor -- start 2023-11-28T11:52:31.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 -- start start 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1d149d0 0x55a6a1cc9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 0x55a6a1e32160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da6c10 0x55a6a1e326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a1da29c0 con 0x55a6a1da2450 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a1cf8f30 con 0x55a6a1da6c10 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1cb03c3080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6a1cf9200 con 0x55a6a1d149d0 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1ca5cc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 0x55a6a1e32160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1ca5cc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 0x55a6a1e32160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36690/0 (socket says 172.21.15.144:36690) 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1ca6cc7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da6c10 0x55a6a1e326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1ca64c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1d149d0 0x55a6a1cc9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.887+0000 7f1ca5cc5700 1 -- 172.21.15.144:0/541784619 learned_addr learned my addr 172.21.15.144:0/541784619 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca5cc5700 1 -- 172.21.15.144:0/541784619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1d149d0 msgr2=0x55a6a1cc9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca5cc5700 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1d149d0 0x55a6a1cc9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca5cc5700 1 -- 172.21.15.144:0/541784619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da6c10 msgr2=0x55a6a1e326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca5cc5700 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da6c10 0x55a6a1e326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca5cc5700 1 -- 172.21.15.144:0/541784619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98005de0 con 0x55a6a1da2450 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca5cc5700 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 0x55a6a1e32160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009860 tx=0x7f1c9c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb8ac43378bda58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca4cc3700 1 -- 172.21.15.144:0/541784619 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00f040 con 0x55a6a1da2450 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca4cc3700 1 -- 172.21.15.144:0/541784619 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c008930 con 0x55a6a1da2450 2023-11-28T11:52:31.891 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.888+0000 7f1ca4cc3700 1 -- 172.21.15.144:0/541784619 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c008ce0 con 0x55a6a1da2450 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 -- 172.21.15.144:0/541784619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 msgr2=0x55a6a1e32160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 0x55a6a1e32160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009860 tx=0x7f1c9c005a00 comp rx=0 tx=0).stop 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 -- 172.21.15.144:0/541784619 shutdown_connections 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da6c10 0x55a6a1e326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1d149d0 0x55a6a1cc9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/541784619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1da2450 0x55a6a1e32160 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 -- 172.21.15.144:0/541784619 >> 172.21.15.144:0/541784619 conn(0x55a6a1cf6b60 msgr2=0x55a6a1da1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 -- 172.21.15.144:0/541784619 shutdown_connections 2023-11-28T11:52:31.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 -- 172.21.15.144:0/541784619 wait complete. 2023-11-28T11:52:31.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 Processor -- start 2023-11-28T11:52:31.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.889+0000 7f1cb03c3080 1 -- start start 2023-11-28T11:52:31.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1cb03c3080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1d149d0 0x55a6a1da3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1cb03c3080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da2450 0x55a6a1d91340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1cb03c3080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1da6c10 0x55a6a1d91880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1cb03c3080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a1d91dc0 con 0x55a6a1d149d0 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1cb03c3080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a1d0d190 con 0x55a6a1da2450 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1cb03c3080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6a1d973c0 con 0x55a6a1da6c10 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1ca64c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1d149d0 0x55a6a1da3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1ca64c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1d149d0 0x55a6a1da3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36694/0 (socket says 172.21.15.144:36694) 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1ca64c6700 1 -- 172.21.15.144:0/2718131240 learned_addr learned my addr 172.21.15.144:0/2718131240 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1ca5cc5700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da2450 0x55a6a1d91340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.890+0000 7f1ca6cc7700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1da6c10 0x55a6a1d91880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1ca64c6700 1 -- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1da6c10 msgr2=0x55a6a1d91880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1ca64c6700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a6a1da6c10 0x55a6a1d91880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1ca64c6700 1 -- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da2450 msgr2=0x55a6a1d91340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1ca64c6700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6a1da2450 0x55a6a1d91340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1ca64c6700 1 -- 172.21.15.144:0/2718131240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c009490 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1ca64c6700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1d149d0 0x55a6a1da3ed0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800a4f0 tx=0x7f1c98002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1c977fe700 1 -- 172.21.15.144:0/2718131240 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98008690 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1c977fe700 1 -- 172.21.15.144:0/2718131240 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98012820 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a6a1c374f0 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1c977fe700 1 -- 172.21.15.144:0/2718131240 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98012bd0 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.891+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a1c1d120 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.892+0000 7f1c977fe700 1 -- 172.21.15.144:0/2718131240 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c98012d30 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1c977fe700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1c88066c10 0x7f1c880690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1c977fe700 1 -- 172.21.15.144:0/2718131240 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c98086730 con 0x55a6a1d149d0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1ca5cc5700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1c88066c10 0x7f1c880690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55a6a1da4410 0x55a6a1d0e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55a6a1d96ec0 con 0x55a6a1da4410 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1ca5cc5700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1c88066c10 0x7f1c880690c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009860 tx=0x7f1c9c012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.893+0000 7f1ca6cc7700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55a6a1da4410 0x55a6a1d0e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1ca6cc7700 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55a6a1da4410 0x55a6a1d0e0f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1ca6cc7700 1 -- 172.21.15.144:0/2718131240 <== osd.5 v2:172.21.15.177:6817/103774 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f1ca000e970 con 0x55a6a1da4410 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55a6a1da4410 msgr2=0x55a6a1d0e0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55a6a1da4410 0x55a6a1d0e0f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1c88066c10 msgr2=0x7f1c880690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1c88066c10 0x7f1c880690c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009860 tx=0x7f1c9c012000 comp rx=0 tx=0).stop 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1d149d0 msgr2=0x55a6a1da3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 --2- 172.21.15.144:0/2718131240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6a1d149d0 0x55a6a1da3ed0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800a4f0 tx=0x7f1c98002fb0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1ca6cc7700 1 -- 172.21.15.144:0/2718131240 reap_dead start 2023-11-28T11:52:31.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 shutdown_connections 2023-11-28T11:52:31.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 >> 172.21.15.144:0/2718131240 conn(0x55a6a1cf6b60 msgr2=0x55a6a1cf8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:31.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 shutdown_connections 2023-11-28T11:52:31.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.894+0000 7f1cb03c3080 1 -- 172.21.15.144:0/2718131240 wait complete. 2023-11-28T11:52:31.900 INFO:teuthology.orchestra.run.smithi144.stdout:mytag 2023-11-28T11:52:31.901 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T11:52:31.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 Processor -- start 2023-11-28T11:52:31.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 -- start start 2023-11-28T11:52:31.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 0x55594e269370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e25e100 0x55594e21d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2f4300 0x55594e25e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594e159fe0 con 0x55594e2205f0 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594e21e6c0 con 0x55594e2f4300 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.944+0000 7f19b8534080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55594e260a70 con 0x55594e25e100 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 0x55594e269370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 0x55594e269370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36710/0 (socket says 172.21.15.144:36710) 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 -- 172.21.15.144:0/1840186728 learned_addr learned my addr 172.21.15.144:0/1840186728 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ade36700 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e25e100 0x55594e21d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19aee38700 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2f4300 0x55594e25e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 -- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e25e100 msgr2=0x55594e21d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e25e100 0x55594e21d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 -- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2f4300 msgr2=0x55594e25e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2f4300 0x55594e25e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 -- 172.21.15.144:0/1840186728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0005de0 con 0x55594e2205f0 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ae637700 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 0x55594e269370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a8f0 tx=0x7f19a000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a8b04794f8e9a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ace34700 1 -- 172.21.15.144:0/1840186728 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0005580 con 0x55594e2205f0 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ace34700 1 -- 172.21.15.144:0/1840186728 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0008b40 con 0x55594e2205f0 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.945+0000 7f19ace34700 1 -- 172.21.15.144:0/1840186728 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0023440 con 0x55594e2205f0 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 -- 172.21.15.144:0/1840186728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 msgr2=0x55594e269370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 0x55594e269370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a8f0 tx=0x7f19a000d000 comp rx=0 tx=0).stop 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 -- 172.21.15.144:0/1840186728 shutdown_connections 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e2205f0 0x55594e269370 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2f4300 0x55594e25e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 --2- 172.21.15.144:0/1840186728 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e25e100 0x55594e21d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 -- 172.21.15.144:0/1840186728 >> 172.21.15.144:0/1840186728 conn(0x55594e24bb60 msgr2=0x55594e37d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 -- 172.21.15.144:0/1840186728 shutdown_connections 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 -- 172.21.15.144:0/1840186728 wait complete. 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 Processor -- start 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.946+0000 7f19b8534080 1 -- start start 2023-11-28T11:52:31.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19b8534080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2205f0 0x55594e262cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19b8534080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e25e100 0x55594e2f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19b8534080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e2f4300 0x55594e37d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19b8534080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594e21f250 con 0x55594e25e100 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19b8534080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55594e37ec60 con 0x55594e2205f0 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19b8534080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55594e21f4a0 con 0x55594e2f4300 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e25e100 0x55594e2f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e25e100 0x55594e2f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36718/0 (socket says 172.21.15.144:36718) 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 -- 172.21.15.144:0/2143732699 learned_addr learned my addr 172.21.15.144:0/2143732699 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:31.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ae637700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2205f0 0x55594e262cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19aee38700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e2f4300 0x55594e37d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 -- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e2f4300 msgr2=0x55594e37d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55594e2f4300 0x55594e37d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 -- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2205f0 msgr2=0x55594e262cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55594e2205f0 0x55594e262cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.947+0000 7f19ade36700 1 -- 172.21.15.144:0/2143732699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55594e18c4f0 con 0x55594e25e100 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.948+0000 7f19ade36700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e25e100 0x55594e2f1120 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009e90 tx=0x7f19a40086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.948+0000 7f199f7fe700 1 -- 172.21.15.144:0/2143732699 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40145f0 con 0x55594e25e100 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.948+0000 7f199f7fe700 1 -- 172.21.15.144:0/2143732699 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4010040 con 0x55594e25e100 2023-11-28T11:52:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.948+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55594e172120 con 0x55594e25e100 2023-11-28T11:52:31.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.948+0000 7f199f7fe700 1 -- 172.21.15.144:0/2143732699 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4014990 con 0x55594e25e100 2023-11-28T11:52:31.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.948+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55594e165830 con 0x55594e25e100 2023-11-28T11:52:31.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.949+0000 7f199f7fe700 1 -- 172.21.15.144:0/2143732699 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19a4014af0 con 0x55594e25e100 2023-11-28T11:52:31.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.949+0000 7f199f7fe700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1990066c10 0x7f19900690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f199f7fe700 1 -- 172.21.15.144:0/2143732699 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19a40849b0 con 0x55594e25e100 2023-11-28T11:52:31.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f19ae637700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1990066c10 0x7f19900690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f19b8534080 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55594e21d490 0x55594e399a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x55594e21e100 con 0x55594e21d490 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f19ae637700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1990066c10 0x7f19900690c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a8f0 tx=0x7f19a0025040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f19aee38700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55594e21d490 0x55594e399a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.950+0000 7f19aee38700 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55594e21d490 0x55594e399a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19aee38700 1 -- 172.21.15.144:0/2143732699 <== osd.5 v2:172.21.15.177:6817/103774 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f19a8006020 con 0x55594e21d490 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55594e21d490 msgr2=0x55594e399a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x55594e21d490 0x55594e399a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1990066c10 msgr2=0x7f19900690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1990066c10 0x7f19900690c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a8f0 tx=0x7f19a0025040 comp rx=0 tx=0).stop 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e25e100 msgr2=0x55594e2f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 --2- 172.21.15.144:0/2143732699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55594e25e100 0x55594e2f1120 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009e90 tx=0x7f19a40086e0 comp rx=0 tx=0).stop 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19aee38700 1 -- 172.21.15.144:0/2143732699 reap_dead start 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 shutdown_connections 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 >> 172.21.15.144:0/2143732699 conn(0x55594e24bb60 msgr2=0x55594e2f3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 shutdown_connections 2023-11-28T11:52:31.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:31.951+0000 7f19b8534080 1 -- 172.21.15.144:0/2143732699 wait complete. 2023-11-28T11:52:31.960 INFO:teuthology.orchestra.run.smithi144.stdout:mytag 2023-11-28T11:52:31.960 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T11:52:32.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.003+0000 7fb9adf1c080 1 Processor -- start 2023-11-28T11:52:32.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.003+0000 7fb9adf1c080 1 -- start start 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9adf1c080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 0x5626e329d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9adf1c080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626e332a0f0 0x5626e325be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9adf1c080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e325c360 0x5626e328b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9adf1c080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e3196fe0 con 0x7fb98c002260 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9adf1c080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e329cd20 con 0x5626e325c360 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9adf1c080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626e332af50 con 0x5626e332a0f0 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 0x5626e329d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 0x5626e329d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36738/0 (socket says 172.21.15.144:36738) 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 -- 172.21.15.144:0/1879038740 learned_addr learned my addr 172.21.15.144:0/1879038740 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a381e700 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626e332a0f0 0x5626e325be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a4820700 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e325c360 0x5626e328b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 -- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626e332a0f0 msgr2=0x5626e325be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626e332a0f0 0x5626e325be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 -- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e325c360 msgr2=0x5626e328b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e325c360 0x5626e328b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 -- 172.21.15.144:0/1879038740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626e31a2830 con 0x7fb98c002260 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.004+0000 7fb9a401f700 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 0x5626e329d1b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb994009910 tx=0x7fb994006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11fc352542524c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9a281c700 1 -- 172.21.15.144:0/1879038740 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9940095e0 con 0x7fb98c002260 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9a281c700 1 -- 172.21.15.144:0/1879038740 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99400bbd0 con 0x7fb98c002260 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9a281c700 1 -- 172.21.15.144:0/1879038740 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994001430 con 0x7fb98c002260 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 -- 172.21.15.144:0/1879038740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 msgr2=0x5626e329d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 0x5626e329d1b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb994009910 tx=0x7fb994006000 comp rx=0 tx=0).stop 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 -- 172.21.15.144:0/1879038740 shutdown_connections 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c002260 0x5626e329d1b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e325c360 0x5626e328b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/1879038740 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5626e332a0f0 0x5626e325be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 -- 172.21.15.144:0/1879038740 >> 172.21.15.144:0/1879038740 conn(0x5626e3288b60 msgr2=0x5626e3331c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 -- 172.21.15.144:0/1879038740 shutdown_connections 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.005+0000 7fb9adf1c080 1 -- 172.21.15.144:0/1879038740 wait complete. 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.006+0000 7fb9adf1c080 1 Processor -- start 2023-11-28T11:52:32.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.006+0000 7fb9adf1c080 1 -- start start 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.006+0000 7fb9adf1c080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e325c360 0x5626e33292a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.006+0000 7fb9adf1c080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e332a0f0 0x5626e33339d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9adf1c080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c002260 0x5626e32a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9adf1c080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e32a0240 con 0x5626e325c360 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9adf1c080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e329f630 con 0x5626e332a0f0 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9adf1c080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626e329f910 con 0x7fb98c002260 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e325c360 0x5626e33292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e325c360 0x5626e33292a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36744/0 (socket says 172.21.15.144:36744) 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 -- 172.21.15.144:0/618117313 learned_addr learned my addr 172.21.15.144:0/618117313 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a381e700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e332a0f0 0x5626e33339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a4820700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c002260 0x5626e32a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 -- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c002260 msgr2=0x5626e32a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c002260 0x5626e32a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 -- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e332a0f0 msgr2=0x5626e33339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e332a0f0 0x5626e33339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 -- 172.21.15.144:0/618117313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c001e10 con 0x5626e325c360 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a381e700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626e332a0f0 0x5626e33339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a4820700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c002260 0x5626e32a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a401f700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e325c360 0x5626e33292a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb994009b70 tx=0x7fb994000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a181a700 1 -- 172.21.15.144:0/618117313 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99402b070 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a181a700 1 -- 172.21.15.144:0/618117313 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99400e470 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626e31af360 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9a181a700 1 -- 172.21.15.144:0/618117313 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99400e820 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.007+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626e31af120 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.008+0000 7fb9a181a700 1 -- 172.21.15.144:0/618117313 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb994021070 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9a181a700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb990066c10 0x7fb9900690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9a181a700 1 -- 172.21.15.144:0/618117313 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb994086dd0 con 0x5626e325c360 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9a381e700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb990066c10 0x7fb9900690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x5626e329f150 0x5626e33cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x5626e32a0c10 con 0x5626e329f150 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9a381e700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb990066c10 0x7fb9900690c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5626e325ea20 tx=0x7fb99800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.009+0000 7fb9a4820700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x5626e329f150 0x5626e33cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9a4820700 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x5626e329f150 0x5626e33cab30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9a4820700 1 -- 172.21.15.144:0/618117313 <== osd.5 v2:172.21.15.177:6817/103774 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fb99c011790 con 0x5626e329f150 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x5626e329f150 msgr2=0x5626e33cab30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x5626e329f150 0x5626e33cab30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb990066c10 msgr2=0x7fb9900690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb990066c10 0x7fb9900690c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5626e325ea20 tx=0x7fb99800c040 comp rx=0 tx=0).stop 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e325c360 msgr2=0x5626e33292a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 --2- 172.21.15.144:0/618117313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626e325c360 0x5626e33292a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb994009b70 tx=0x7fb994000fe0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9a4820700 1 -- 172.21.15.144:0/618117313 reap_dead start 2023-11-28T11:52:32.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 shutdown_connections 2023-11-28T11:52:32.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 >> 172.21.15.144:0/618117313 conn(0x5626e3288b60 msgr2=0x5626e33386d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:32.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 shutdown_connections 2023-11-28T11:52:32.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.010+0000 7fb9adf1c080 1 -- 172.21.15.144:0/618117313 wait complete. 2023-11-28T11:52:32.017 INFO:teuthology.orchestra.run.smithi144.stdout:mytag 2023-11-28T11:52:32.018 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T11:52:32.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 Processor -- start 2023-11-28T11:52:32.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 -- start start 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52cb3190 0x556e52cb3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52c0f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 0x556e52c094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e52b88000 con 0x556e52c09030 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e52b1afe0 con 0x556e52c0fbd0 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c165dc080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e52c106f0 con 0x556e52cb3190 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.101+0000 7f4c0cee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 0x556e52c094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 0x556e52c094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36770/0 (socket says 172.21.15.144:36770) 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 -- 172.21.15.144:0/3947617291 learned_addr learned my addr 172.21.15.144:0/3947617291 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0bede700 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52c0f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0c6df700 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52cb3190 0x556e52cb3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 -- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52cb3190 msgr2=0x556e52cb3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52cb3190 0x556e52cb3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 -- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 msgr2=0x556e52c0f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52c0f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 -- 172.21.15.144:0/3947617291 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e52b33360 con 0x556e52c09030 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0cee0700 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 0x556e52c094d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c04001330 tx=0x7f4c04004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea2e4cf162f7548 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0aedc700 1 -- 172.21.15.144:0/3947617291 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04001d10 con 0x556e52c09030 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0aedc700 1 -- 172.21.15.144:0/3947617291 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04001e70 con 0x556e52c09030 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c0aedc700 1 -- 172.21.15.144:0/3947617291 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04006b10 con 0x556e52c09030 2023-11-28T11:52:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c165dc080 1 -- 172.21.15.144:0/3947617291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 msgr2=0x556e52c094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c165dc080 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 0x556e52c094d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c04001330 tx=0x7f4c04004140 comp rx=0 tx=0).stop 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c165dc080 1 -- 172.21.15.144:0/3947617291 shutdown_connections 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c165dc080 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52c09030 0x556e52c094d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c165dc080 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52cb3190 0x556e52cb3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.102+0000 7f4c165dc080 1 --2- 172.21.15.144:0/3947617291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52c0f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.103+0000 7f4c165dc080 1 -- 172.21.15.144:0/3947617291 >> 172.21.15.144:0/3947617291 conn(0x556e52c28330 msgr2=0x556e52c084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.103+0000 7f4c165dc080 1 -- 172.21.15.144:0/3947617291 shutdown_connections 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.103+0000 7f4c165dc080 1 -- 172.21.15.144:0/3947617291 wait complete. 2023-11-28T11:52:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.103+0000 7f4c165dc080 1 Processor -- start 2023-11-28T11:52:32.107 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.103+0000 7f4c165dc080 1 -- start start 2023-11-28T11:52:32.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c165dc080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52c09030 0x556e52caa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c165dc080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52be0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c165dc080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52cb3190 0x556e52be1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c165dc080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e52bf8f70 con 0x556e52cb3190 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c165dc080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e52cafbd0 con 0x556e52c0fbd0 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c165dc080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e52bf89b0 con 0x556e52c09030 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52cb3190 0x556e52be1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52cb3190 0x556e52be1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36776/0 (socket says 172.21.15.144:36776) 2023-11-28T11:52:32.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0c6df700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52c09030 0x556e52caa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 -- 172.21.15.144:0/2335285578 learned_addr learned my addr 172.21.15.144:0/2335285578 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0bede700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52be0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 -- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52c09030 msgr2=0x556e52caa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x556e52c09030 0x556e52caa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 -- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 msgr2=0x556e52be0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e52c0fbd0 0x556e52be0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 -- 172.21.15.144:0/2335285578 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0400e040 con 0x556e52cb3190 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.104+0000 7f4c0cee0700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52cb3190 0x556e52be1500 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x556e52c119f0 tx=0x7f4c04001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.105+0000 7f4c09eda700 1 -- 172.21.15.144:0/2335285578 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04001d10 con 0x556e52cb3190 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.105+0000 7f4c09eda700 1 -- 172.21.15.144:0/2335285578 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04003630 con 0x556e52cb3190 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.105+0000 7f4c09eda700 1 -- 172.21.15.144:0/2335285578 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04013450 con 0x556e52cb3190 2023-11-28T11:52:32.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.105+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556e52c2a4f0 con 0x556e52cb3190 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.105+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e52b26830 con 0x556e52cb3190 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.106+0000 7f4c09eda700 1 -- 172.21.15.144:0/2335285578 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c040135b0 con 0x556e52cb3190 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.106+0000 7f4c09eda700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4bf8066c10 0x7f4bf80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.106+0000 7f4c09eda700 1 -- 172.21.15.144:0/2335285578 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c04085dd0 con 0x556e52cb3190 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.106+0000 7f4c0c6df700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4bf8066c10 0x7f4bf80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.106+0000 7f4c165dc080 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x556e52cade40 0x556e52cb0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.106+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 --> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) v8 -- 0x556e52cb1070 con 0x556e52cade40 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c0bede700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x556e52cade40 0x556e52cb0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c0c6df700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4bf8066c10 0x7f4bf80690c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c00000c00 tx=0x7f4c0000f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c0bede700 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x556e52cade40 0x556e52cb0b30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c0bede700 1 -- 172.21.15.144:0/2335285578 <== osd.7 v2:172.21.15.177:6816/103773 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4bfc009a30 con 0x556e52cade40 2023-11-28T11:52:32.113 INFO:teuthology.orchestra.run.smithi144.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x556e52cade40 msgr2=0x556e52cb0b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x556e52cade40 0x556e52cb0b30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4bf8066c10 msgr2=0x7f4bf80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4bf8066c10 0x7f4bf80690c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c00000c00 tx=0x7f4c0000f000 comp rx=0 tx=0).stop 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52cb3190 msgr2=0x556e52be1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 --2- 172.21.15.144:0/2335285578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e52cb3190 0x556e52be1500 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x556e52c119f0 tx=0x7f4c04001650 comp rx=0 tx=0).stop 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c0cee0700 1 -- 172.21.15.144:0/2335285578 reap_dead start 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 shutdown_connections 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.107+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 >> 172.21.15.144:0/2335285578 conn(0x556e52c28330 msgr2=0x556e52c0f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.108+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 shutdown_connections 2023-11-28T11:52:32.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.108+0000 7f4c165dc080 1 -- 172.21.15.144:0/2335285578 wait complete. 2023-11-28T11:52:32.115 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:52:32.115 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:52:32.198 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:52:32.245 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:52:32.245 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:32.245 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:32.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:52:32.263 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:52:32.263 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:32.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:52:32.283 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:32.283 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:32.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:52:32.320 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:52:32.320 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:32.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:52:32.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:32.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.478+0000 7fe014880700 1 Processor -- start 2023-11-28T11:52:32.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.478+0000 7fe014880700 1 -- start start 2023-11-28T11:52:32.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe014880700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe01006dee0 0x7fe01006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe014880700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01015d060 con 0x7fe010071040 2023-11-28T11:52:32.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe014880700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01015d340 con 0x7fe010160540 2023-11-28T11:52:32.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe014880700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01015d620 con 0x7fe01006dee0 2023-11-28T11:52:32.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe00e59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe010071040 0x7fe010071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53494/0 (socket says 172.21.15.144:53494) 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe00e59c700 1 -- 172.21.15.144:0/406821162 learned_addr learned my addr 172.21.15.144:0/406821162 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949633624 0 0) 0x7fe01015d060 con 0x7fe010071040 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80035b0 con 0x7fe010071040 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.479+0000 7fe00dd9b700 1 --2- 172.21.15.144:0/406821162 >> v2:172.21.15.177:3300/0 conn(0x7fe01006dee0 0x7fe01006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681558277 0 0) 0x7fdff80035b0 con 0x7fe010071040 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 >> v2:172.21.15.177:3300/0 conn(0x7fe01006dee0 msgr2=0x7fe01006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 --2- 172.21.15.144:0/406821162 >> v2:172.21.15.177:3300/0 conn(0x7fe01006dee0 0x7fe01006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 >> v1:172.21.15.177:6789/0 conn(0x7fe010160540 legacy=0x7fe010160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01015d900 con 0x7fe010071040 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fe004001670 con 0x7fe010071040 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 >> v1:172.21.15.144:6789/0 conn(0x7fe010071040 legacy=0x7fe010071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff8003a00 0x7fdff8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdff8004850 0x7fdff8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fdff800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01015d620 con 0x7fdff8005200 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff800c190 con 0x7fdff8003a00 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff800c350 con 0x7fdff8004850 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fdff800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00e59c700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff8003a00 0x7fdff8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00dd9b700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdff8004850 0x7fdff8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 -- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdff8004850 msgr2=0x7fdff8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdff8004850 0x7fdff8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 -- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff8003a00 msgr2=0x7fdff8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:32.485 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdff8003a00 0x7fdff8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 -- 172.21.15.144:0/406821162 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80035b0 con 0x7fdff8005200 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 -- 172.21.15.144:0/406821162 reap_dead start 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.480+0000 7fe00ed9d700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fdff800ba10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c210 tx=0x7fdff8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603b1c65109f8f26 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008016070 con 0x7fdff8005200 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800cbb0 con 0x7fdff8005200 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe00d59a700 1 -- 172.21.15.144:0/406821162 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008001e70 con 0x7fdff8005200 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 -- 172.21.15.144:0/406821162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 msgr2=0x7fdff800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fdff800ba10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c210 tx=0x7fdff8005880 comp rx=0 tx=0).stop 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 -- 172.21.15.144:0/406821162 shutdown_connections 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 --2- 172.21.15.144:0/406821162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fdff800ba10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 -- 172.21.15.144:0/406821162 >> 172.21.15.144:0/406821162 conn(0x7fe0100d0000 msgr2=0x7fe0100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 -- 172.21.15.144:0/406821162 shutdown_connections 2023-11-28T11:52:32.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.481+0000 7fe014880700 1 -- 172.21.15.144:0/406821162 wait complete. 2023-11-28T11:52:32.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.482+0000 7fe014880700 1 Processor -- start 2023-11-28T11:52:32.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 -- start start 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fe0101f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0100b4720 0x7fe0101f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101f1f30 0x7fe0101f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f33c0 con 0x7fdff8005200 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b4bc0 con 0x7fe0101f1f30 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe014880700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b4ea0 con 0x7fe0100b4720 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe00e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fe0101f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe00e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fe0101f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36794/0 (socket says 172.21.15.144:36794) 2023-11-28T11:52:32.490 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe00e59c700 1 -- 172.21.15.144:0/4016176454 learned_addr learned my addr 172.21.15.144:0/4016176454 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:32.491 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe00ed9d700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101f1f30 0x7fe0101f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.491 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.483+0000 7fe00dd9b700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0100b4720 0x7fe0101f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.491 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe00e59c700 1 -- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0100b4720 msgr2=0x7fe0101f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe00e59c700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0100b4720 0x7fe0101f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe00e59c700 1 -- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101f1f30 msgr2=0x7fe0101f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe00e59c700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101f1f30 0x7fe0101f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe00e59c700 1 -- 172.21.15.144:0/4016176454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01006a270 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe00e59c700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fe0101f2e80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe004001190 tx=0x7fe004007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004012560 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0040126e0 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01015ccd0 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004012a90 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.484+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0100b5460 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.485+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01006a400 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.485+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe004012bf0 con 0x7fdff8005200 2023-11-28T11:52:32.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.486+0000 7fdffeffd700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdff00682f0 0x7fdff006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:32.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.486+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe004085d80 con 0x7fdff8005200 2023-11-28T11:52:32.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.486+0000 7fe00dd9b700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdff00682f0 0x7fdff006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:32.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.486+0000 7fe00dd9b700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdff00682f0 0x7fdff006a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0100b4af0 tx=0x7fe000009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:32.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.490+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe01006a400 con 0x7fdff8005200 2023-11-28T11:52:32.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.714+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe01006a400 con 0x7fdff8005200 2023-11-28T11:52:34.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.714+0000 7fdffeffd700 1 -- 172.21.15.144:0/4016176454 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fe01006a400 con 0x7fdff8005200 2023-11-28T11:52:34.663 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:34.663 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdff00682f0 msgr2=0x7fdff006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdff00682f0 0x7fdff006a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0100b4af0 tx=0x7fe000009210 comp rx=0 tx=0).stop 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 msgr2=0x7fe0101f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fe0101f2e80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe004001190 tx=0x7fe004007340 comp rx=0 tx=0).stop 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 shutdown_connections 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff8005200 0x7fe0101f2e80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdff00682f0 0x7fdff006a7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0101f1f30 0x7fe0101f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 --2- 172.21.15.144:0/4016176454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe0100b4720 0x7fe0101f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 >> 172.21.15.144:0/4016176454 conn(0x7fe0100d0000 msgr2=0x7fe01015e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:34.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 shutdown_connections 2023-11-28T11:52:34.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:32.717+0000 7fe014880700 1 -- 172.21.15.144:0/4016176454 wait complete. 2023-11-28T11:52:34.666 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:34.666 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:34.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.803+0000 7f5063a2d700 1 Processor -- start 2023-11-28T11:52:34.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.803+0000 7f5063a2d700 1 -- start start 2023-11-28T11:52:34.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5063a2d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5063a2d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 0x7f505c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5063a2d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c15d060 con 0x7f505c071040 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5063a2d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15d340 con 0x7f505c160540 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5063a2d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15d620 con 0x7f505c06dee0 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5060fc8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5061fca700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 0x7f505c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5061fca700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 0x7f505c1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:39080/0 (socket says 172.21.15.144:39080) 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5060fc8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:48884/0 (socket says 172.21.15.144:48884) 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f5061fca700 1 -- 172.21.15.144:0/2696258104 learned_addr learned my addr 172.21.15.144:0/2696258104 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.804+0000 7f50617c9700 1 --1- 172.21.15.144:0/2696258104 >> v1:172.21.15.177:6789/0 conn(0x7f505c071040 0x7f505c071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:54004/0 (socket says 172.21.15.144:54004) 2023-11-28T11:52:34.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5060fc8700 1 -- 172.21.15.144:0/2696258104 >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 msgr2=0x7f505c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5060fc8700 1 --2- 172.21.15.144:0/2696258104 >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 0x7f505c1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5060fc8700 1 -- 172.21.15.144:0/2696258104 >> v1:172.21.15.177:6789/0 conn(0x7f505c071040 legacy=0x7f505c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5060fc8700 1 -- 172.21.15.144:0/2696258104 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c15d900 con 0x7f505c06dee0 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5061fca700 1 --2- 172.21.15.144:0/2696258104 >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 0x7f505c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5060fc8700 1 --2- 172.21.15.144:0/2696258104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c06e330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f504c009840 tx=0x7f504c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d670adbaa57c13dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5053fff700 1 -- 172.21.15.144:0/2696258104 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011830 con 0x7f505c06dee0 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5053fff700 1 -- 172.21.15.144:0/2696258104 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c011990 con 0x7f505c06dee0 2023-11-28T11:52:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5053fff700 1 -- 172.21.15.144:0/2696258104 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c011d20 con 0x7f505c06dee0 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5063a2d700 1 -- 172.21.15.144:0/2696258104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 msgr2=0x7f505c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.805+0000 7f5063a2d700 1 --2- 172.21.15.144:0/2696258104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c06e330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f504c009840 tx=0x7f504c009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.806+0000 7f5063a2d700 1 -- 172.21.15.144:0/2696258104 shutdown_connections 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.806+0000 7f5063a2d700 1 --2- 172.21.15.144:0/2696258104 >> v2:172.21.15.177:3300/0 conn(0x7f505c160540 0x7f505c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.806+0000 7f5063a2d700 1 --2- 172.21.15.144:0/2696258104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c06e330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.806+0000 7f5063a2d700 1 -- 172.21.15.144:0/2696258104 >> 172.21.15.144:0/2696258104 conn(0x7f505c0d0000 msgr2=0x7f505c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.806+0000 7f5063a2d700 1 -- 172.21.15.144:0/2696258104 shutdown_connections 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.806+0000 7f5063a2d700 1 -- 172.21.15.144:0/2696258104 wait complete. 2023-11-28T11:52:34.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.807+0000 7f5063a2d700 1 Processor -- start 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.807+0000 7f5063a2d700 1 -- start start 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c1485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 0x7f505c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f505c160540 0x7f505c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0b2e30 con 0x7f505c071040 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15b780 con 0x7f505c160540 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15ba60 con 0x7f505c06dee0 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 0x7f505c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 0x7f505c15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36810/0 (socket says 172.21.15.144:36810) 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 -- 172.21.15.144:0/176589435 learned_addr learned my addr 172.21.15.144:0/176589435 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5061fca700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f505c160540 0x7f505c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f50617c9700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c1485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 -- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 msgr2=0x7f505c1485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c1485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 -- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f505c160540 msgr2=0x7f505c0adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f505c160540 0x7f505c0adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 -- 172.21.15.144:0/176589435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054009b30 con 0x7f505c071040 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f50617c9700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c1485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5060fc8700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 0x7f505c15ace0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f504c0090b0 tx=0x7f504c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0036a0 con 0x7f505c071040 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504c01e070 con 0x7f505c071040 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c0094a0 con 0x7f505c071040 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.808+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c003a60 con 0x7f505c071040 2023-11-28T11:52:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.809+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c15ccd0 con 0x7f505c071040 2023-11-28T11:52:34.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.810+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c06e870 con 0x7f505c071040 2023-11-28T11:52:34.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.810+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f504c00f050 con 0x7f505c071040 2023-11-28T11:52:34.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.810+0000 7f5051ffb700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50400682f0 0x7f504006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.810+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f504c086b30 con 0x7f505c071040 2023-11-28T11:52:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.810+0000 7f50617c9700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50400682f0 0x7f504006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.811+0000 7f50617c9700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50400682f0 0x7f504006a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5058000f60 tx=0x7f5058007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.815+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f504c053390 con 0x7f505c071040 2023-11-28T11:52:34.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.997+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f505c06a270 con 0x7f505c071040 2023-11-28T11:52:34.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:34.998+0000 7f5051ffb700 1 -- 172.21.15.144:0/176589435 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f504c020090 con 0x7f505c071040 2023-11-28T11:52:35.000 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:35.000 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:10.310687+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50400682f0 msgr2=0x7f504006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50400682f0 0x7f504006a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5058000f60 tx=0x7f5058007dd0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 msgr2=0x7f505c15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 0x7f505c15ace0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f504c0090b0 tx=0x7f504c004750 comp rx=0 tx=0).stop 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 shutdown_connections 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50400682f0 0x7f504006a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f505c160540 0x7f505c0adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f505c06dee0 0x7f505c1485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 --2- 172.21.15.144:0/176589435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f505c071040 0x7f505c15ace0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 >> 172.21.15.144:0/176589435 conn(0x7f505c0d0000 msgr2=0x7f505c15e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 shutdown_connections 2023-11-28T11:52:35.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.001+0000 7f5063a2d700 1 -- 172.21.15.144:0/176589435 wait complete. 2023-11-28T11:52:35.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:52:07.570239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:52:08.264687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:52:35.014 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:52:35.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.135+0000 7ffa5eb37700 1 Processor -- start 2023-11-28T11:52:35.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.135+0000 7ffa5eb37700 1 -- start start 2023-11-28T11:52:35.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa5eb37700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 0x7ffa580af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa5eb37700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580cc5f0 con 0x7ffa580cbdf0 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa5eb37700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5815d470 con 0x7ffa5806ba90 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa5eb37700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5815d750 con 0x7ffa5815cef0 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa57fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ffa580cbdf0 0x7ffa5806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53528/0 (socket says 172.21.15.144:53528) 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa57fff700 1 -- 172.21.15.144:0/1675624011 learned_addr learned my addr 172.21.15.144:0/1675624011 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa5c8d3700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 0x7ffa580af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437698027 0 0) 0x7ffa580cc5f0 con 0x7ffa580cbdf0 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.136+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa480035b0 con 0x7ffa580cbdf0 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993734513 0 0) 0x7ffa5815d470 con 0x7ffa5806ba90 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa580cc5f0 con 0x7ffa5806ba90 2023-11-28T11:52:35.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843630431 0 0) 0x7ffa480035b0 con 0x7ffa580cbdf0 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 msgr2=0x7ffa580af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 0x7ffa580af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 >> v1:172.21.15.177:6789/0 conn(0x7ffa5806ba90 legacy=0x7ffa580ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5815da30 con 0x7ffa580cbdf0 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7ffa44001670 con 0x7ffa580cbdf0 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 >> v1:172.21.15.144:6789/0 conn(0x7ffa580cbdf0 legacy=0x7ffa5806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 0x7ffa48003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa48004170 0x7ffa48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5815d750 con 0x7ffa48004d90 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4800bb20 con 0x7ffa48004170 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4800bce0 con 0x7ffa5815cef0 2023-11-28T11:52:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa57fff700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa48004170 0x7ffa48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5c8d3700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 0x7ffa48003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 -- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 msgr2=0x7ffa48003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa5815cef0 0x7ffa48003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 -- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa48004170 msgr2=0x7ffa48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa48004170 0x7ffa48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 -- 172.21.15.144:0/1675624011 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480035b0 con 0x7ffa48004d90 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa5d0d4700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa48005200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa50002050 tx=0x7ffa48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83b2118230a36d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000ce60 con 0x7ffa48004d90 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50003a00 con 0x7ffa48004d90 2023-11-28T11:52:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.137+0000 7ffa577fe700 1 -- 172.21.15.144:0/1675624011 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50003db0 con 0x7ffa48004d90 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5eb37700 1 -- 172.21.15.144:0/1675624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 msgr2=0x7ffa48005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/1675624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa48005200 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa50002050 tx=0x7ffa48005880 comp rx=0 tx=0).stop 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5d0d4700 1 -- 172.21.15.144:0/1675624011 reap_dead start 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5eb37700 1 -- 172.21.15.144:0/1675624011 shutdown_connections 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5eb37700 1 -- 172.21.15.144:0/1675624011 >> 172.21.15.144:0/1675624011 conn(0x7ffa580d0000 msgr2=0x7ffa580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5eb37700 1 -- 172.21.15.144:0/1675624011 shutdown_connections 2023-11-28T11:52:35.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.138+0000 7ffa5eb37700 1 -- 172.21.15.144:0/1675624011 wait complete. 2023-11-28T11:52:35.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.140+0000 7ffa5eb37700 1 Processor -- start 2023-11-28T11:52:35.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.140+0000 7ffa5eb37700 1 -- start start 2023-11-28T11:52:35.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa581f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa580b55b0 0x7ffa5815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa58151fb0 0x7ffa58152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580c44b0 con 0x7ffa48004d90 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa581f6bb0 con 0x7ffa580b55b0 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580b5b70 con 0x7ffa58151fb0 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa581f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa581f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36828/0 (socket says 172.21.15.144:36828) 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 -- 172.21.15.144:0/908779966 learned_addr learned my addr 172.21.15.144:0/908779966 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 -- 172.21.15.144:0/908779966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa58151fb0 msgr2=0x7ffa58152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa58151fb0 0x7ffa58152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 -- 172.21.15.144:0/908779966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa580b55b0 msgr2=0x7ffa5815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa580b55b0 0x7ffa5815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 -- 172.21.15.144:0/908779966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5815dff0 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5c8d3700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa581f6670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c011a50 tx=0x7ffa4c011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00f940 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c00faa0 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5815dd10 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00fe50 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.141+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa580af400 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.142+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa580c4720 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.143+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffa4c014440 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.143+0000 7ffa557fa700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffa3c0682f0 0x7ffa3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.143+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa4c086650 con 0x7ffa48004d90 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.143+0000 7ffa57fff700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffa3c0682f0 0x7ffa3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.144+0000 7ffa57fff700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffa3c0682f0 0x7ffa3c06a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa440011e0 tx=0x7ffa44001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.147+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffa4c052e50 con 0x7ffa48004d90 2023-11-28T11:52:35.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.329+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ffa580d0e60 con 0x7ffa48004d90 2023-11-28T11:52:35.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.329+0000 7ffa557fa700 1 -- 172.21.15.144:0/908779966 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7ffa4c056470 con 0x7ffa48004d90 2023-11-28T11:52:35.330 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:52:35.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.331+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffa3c0682f0 msgr2=0x7ffa3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.331+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffa3c0682f0 0x7ffa3c06a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa440011e0 tx=0x7ffa44001a30 comp rx=0 tx=0).stop 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.331+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 msgr2=0x7ffa581f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.331+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa581f6670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c011a50 tx=0x7ffa4c011e10 comp rx=0 tx=0).stop 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 shutdown_connections 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa48004d90 0x7ffa581f6670 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ffa3c0682f0 0x7ffa3c06a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffa58151fb0 0x7ffa58152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 --2- 172.21.15.144:0/908779966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa580b55b0 0x7ffa5815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 >> 172.21.15.144:0/908779966 conn(0x7ffa580d0000 msgr2=0x7ffa5806a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 shutdown_connections 2023-11-28T11:52:35.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.332+0000 7ffa5eb37700 1 -- 172.21.15.144:0/908779966 wait complete. 2023-11-28T11:52:35.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:52:35.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.468+0000 7f6cf0796700 1 Processor -- start 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.468+0000 7f6cf0796700 1 -- start start 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6cf0796700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6ce81658c0 0x7f6ce8167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6cf0796700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce815d060 con 0x7f6ce806dee0 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6cf0796700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce815d340 con 0x7f6ce8071040 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6cf0796700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce8161550 con 0x7f6ce81658c0 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ceed33700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6ce81658c0 0x7f6ce8167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6cedd31700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6ce806dee0 0x7f6ce806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53548/0 (socket says 172.21.15.144:53548) 2023-11-28T11:52:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6cedd31700 1 -- 172.21.15.144:0/1786496271 learned_addr learned my addr 172.21.15.144:0/1786496271 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:35.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ceed33700 1 -- 172.21.15.144:0/1786496271 >> v1:172.21.15.177:6789/0 conn(0x7f6ce8071040 legacy=0x7f6ce8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ceed33700 1 -- 172.21.15.144:0/1786496271 >> v1:172.21.15.144:6789/0 conn(0x7f6ce806dee0 legacy=0x7f6ce806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ceed33700 1 -- 172.21.15.144:0/1786496271 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce815d620 con 0x7f6ce81658c0 2023-11-28T11:52:35.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ced530700 1 -- 172.21.15.144:0/1786496271 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668406295 0 0) 0x7f6ce815d060 con 0x7f6ce806dee0 2023-11-28T11:52:35.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ceed33700 1 --2- 172.21.15.144:0/1786496271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce81658c0 0x7f6ce8167ca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009d90 tx=0x7f6ce000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300b80e9c64bb639 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ced530700 1 -- 172.21.15.144:0/1786496271 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000e070 con 0x7f6ce81658c0 2023-11-28T11:52:35.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ced530700 1 -- 172.21.15.144:0/1786496271 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce000bbd0 con 0x7f6ce81658c0 2023-11-28T11:52:35.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.469+0000 7f6ced530700 1 -- 172.21.15.144:0/1786496271 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0012430 con 0x7f6ce81658c0 2023-11-28T11:52:35.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 -- 172.21.15.144:0/1786496271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce81658c0 msgr2=0x7f6ce8167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 --2- 172.21.15.144:0/1786496271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce81658c0 0x7f6ce8167ca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009d90 tx=0x7f6ce000b330 comp rx=0 tx=0).stop 2023-11-28T11:52:35.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 -- 172.21.15.144:0/1786496271 shutdown_connections 2023-11-28T11:52:35.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 --2- 172.21.15.144:0/1786496271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce81658c0 0x7f6ce8167ca0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 -- 172.21.15.144:0/1786496271 >> 172.21.15.144:0/1786496271 conn(0x7f6ce80d0000 msgr2=0x7f6ce80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 -- 172.21.15.144:0/1786496271 shutdown_connections 2023-11-28T11:52:35.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.470+0000 7f6cf0796700 1 -- 172.21.15.144:0/1786496271 wait complete. 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.471+0000 7f6cf0796700 1 Processor -- start 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.471+0000 7f6cf0796700 1 -- start start 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cf0796700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 0x7f6ce80c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cf0796700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8071040 0x7f6ce80acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cf0796700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce81658c0 0x7f6ce8156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cf0796700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80ae090 con 0x7f6ce806dee0 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cf0796700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8157560 con 0x7f6ce8071040 2023-11-28T11:52:35.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cf0796700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80ad480 con 0x7f6ce81658c0 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 0x7f6ce80c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 0x7f6ce80c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36842/0 (socket says 172.21.15.144:36842) 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 -- 172.21.15.144:0/3739551143 learned_addr learned my addr 172.21.15.144:0/3739551143 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 -- 172.21.15.144:0/3739551143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce81658c0 msgr2=0x7f6ce8156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce81658c0 0x7f6ce8156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 -- 172.21.15.144:0/3739551143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8071040 msgr2=0x7f6ce80acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8071040 0x7f6ce80acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.472+0000 7f6cee532700 1 -- 172.21.15.144:0/3739551143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0009a70 con 0x7f6ce806dee0 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.473+0000 7f6cee532700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 0x7f6ce80c44b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c4d0 tx=0x7f6ce400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.473+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40052d0 con 0x7f6ce806dee0 2023-11-28T11:52:35.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.473+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4005430 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.473+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4003880 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.473+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce815ccd0 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.473+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce815d900 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.474+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce806a270 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.474+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ce4019030 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.474+0000 7f6cdeffd700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6ccc0682f0 0x7f6ccc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.475+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6ce4084fb0 con 0x7f6ce806dee0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.475+0000 7f6cedd31700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6ccc0682f0 0x7f6ccc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.475+0000 7f6cedd31700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6ccc0682f0 0x7f6ccc06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8000c00 tx=0x7f6cd8007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.479+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ce404b5e0 con 0x7f6ce806dee0 2023-11-28T11:52:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.659+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ce80ce960 con 0x7f6ce806dee0 2023-11-28T11:52:35.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.659+0000 7f6cdeffd700 1 -- 172.21.15.144:0/3739551143 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ce40170a0 con 0x7f6ce806dee0 2023-11-28T11:52:35.660 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:52:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6ccc0682f0 msgr2=0x7f6ccc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6ccc0682f0 0x7f6ccc06a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8000c00 tx=0x7f6cd8007a50 comp rx=0 tx=0).stop 2023-11-28T11:52:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 msgr2=0x7f6ce80c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 0x7f6ce80c44b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c4d0 tx=0x7f6ce400c890 comp rx=0 tx=0).stop 2023-11-28T11:52:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 shutdown_connections 2023-11-28T11:52:35.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ce806dee0 0x7f6ce80c44b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6ccc0682f0 0x7f6ccc06a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce81658c0 0x7f6ce8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 --2- 172.21.15.144:0/3739551143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8071040 0x7f6ce80acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 >> 172.21.15.144:0/3739551143 conn(0x7f6ce80d0000 msgr2=0x7f6ce80cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 shutdown_connections 2023-11-28T11:52:35.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.661+0000 7f6cf0796700 1 -- 172.21.15.144:0/3739551143 wait complete. 2023-11-28T11:52:35.675 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:52:35.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.799+0000 7fe8e9e07700 1 Processor -- start 2023-11-28T11:52:35.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.799+0000 7fe8e9e07700 1 -- start start 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.799+0000 7fe8e9e07700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe8e4140950 0x7fe8e40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.799+0000 7fe8e9e07700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e413ff00 con 0x7fe8e40b2e70 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.799+0000 7fe8e9e07700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e41404c0 con 0x7fe8e40cd8c0 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.799+0000 7fe8e9e07700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e4140dc0 con 0x7fe8e4140950 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e2ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe8e4140950 0x7fe8e40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e3fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe8e40b2e70 0x7fe8e406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53566/0 (socket says 172.21.15.144:53566) 2023-11-28T11:52:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e3fff700 1 -- 172.21.15.144:0/2168645208 learned_addr learned my addr 172.21.15.144:0/2168645208 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e2ffd700 1 --2- 172.21.15.144:0/2168645208 >> v2:172.21.15.144:3300/0 conn(0x7fe8e4140950 0x7fe8e40b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36852/0 (socket says 172.21.15.144:36852) 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e2ffd700 1 -- 172.21.15.144:0/2168645208 >> v1:172.21.15.177:6789/0 conn(0x7fe8e40cd8c0 legacy=0x7fe8e40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e2ffd700 1 -- 172.21.15.144:0/2168645208 >> v1:172.21.15.144:6789/0 conn(0x7fe8e40b2e70 legacy=0x7fe8e406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e27fc700 1 -- 172.21.15.144:0/2168645208 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101655183 0 0) 0x7fe8e413ff00 con 0x7fe8e40b2e70 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e2ffd700 1 -- 172.21.15.144:0/2168645208 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e415aa80 con 0x7fe8e4140950 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e2ffd700 1 --2- 172.21.15.144:0/2168645208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b2930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009840 tx=0x7fe8d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604b27ee4b29ffd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e27fc700 1 -- 172.21.15.144:0/2168645208 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4004230 con 0x7fe8e4140950 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e27fc700 1 -- 172.21.15.144:0/2168645208 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4004390 con 0x7fe8e4140950 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.800+0000 7fe8e27fc700 1 -- 172.21.15.144:0/2168645208 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40036a0 con 0x7fe8e4140950 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 -- 172.21.15.144:0/2168645208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 msgr2=0x7fe8e40b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/2168645208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b2930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009840 tx=0x7fe8d4009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:35.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 -- 172.21.15.144:0/2168645208 shutdown_connections 2023-11-28T11:52:35.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/2168645208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b2930 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 -- 172.21.15.144:0/2168645208 >> 172.21.15.144:0/2168645208 conn(0x7fe8e406aed0 msgr2=0x7fe8e406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 -- 172.21.15.144:0/2168645208 shutdown_connections 2023-11-28T11:52:35.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.801+0000 7fe8e9e07700 1 -- 172.21.15.144:0/2168645208 wait complete. 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.802+0000 7fe8e9e07700 1 Processor -- start 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.802+0000 7fe8e9e07700 1 -- start start 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e9e07700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e40b2e70 0x7fe8e4154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e9e07700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40cd8c0 0x7fe8e41419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e9e07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e9e07700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e415ae10 con 0x7fe8e4140950 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e9e07700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e415b0f0 con 0x7fe8e40cd8c0 2023-11-28T11:52:35.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e9e07700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e415b3d0 con 0x7fe8e40b2e70 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36860/0 (socket says 172.21.15.144:36860) 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 -- 172.21.15.144:0/3353299224 learned_addr learned my addr 172.21.15.144:0/3353299224 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e2ffd700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40cd8c0 0x7fe8e41419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e37fe700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e40b2e70 0x7fe8e4154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 -- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e40b2e70 msgr2=0x7fe8e4154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e40b2e70 0x7fe8e4154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 -- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40cd8c0 msgr2=0x7fe8e41419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40cd8c0 0x7fe8e41419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e3fff700 1 -- 172.21.15.144:0/3353299224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40094a0 con 0x7fe8e4140950 2023-11-28T11:52:35.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.803+0000 7fe8e2ffd700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40cd8c0 0x7fe8e41419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.804+0000 7fe8e3fff700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b6220 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00cca0 tx=0x7fe8dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.804+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00cea0 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.804+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc00ade0 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.804+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e40b35f0 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.804+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc007310 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.804+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e40b3310 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.805+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e4140150 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.805+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe8dc01a030 con 0x7fe8e4140950 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.806+0000 7fe8dbfff700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8c40682f0 0x7fe8c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.806+0000 7fe8e37fe700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8c40682f0 0x7fe8c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:35.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.806+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8dc085bd0 con 0x7fe8e4140950 2023-11-28T11:52:35.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.806+0000 7fe8e37fe700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8c40682f0 0x7fe8c406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc004f80 tx=0x7fe8cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:35.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.810+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe8dc052340 con 0x7fe8e4140950 2023-11-28T11:52:35.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.992+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8e4068690 con 0x7fe8e4140950 2023-11-28T11:52:35.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.993+0000 7fe8dbfff700 1 -- 172.21.15.144:0/3353299224 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8dc018020 con 0x7fe8e4140950 2023-11-28T11:52:35.993 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8c40682f0 msgr2=0x7fe8c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8c40682f0 0x7fe8c406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc004f80 tx=0x7fe8cc009040 comp rx=0 tx=0).stop 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 msgr2=0x7fe8e40b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b6220 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00cca0 tx=0x7fe8dc00a340 comp rx=0 tx=0).stop 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 shutdown_connections 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8e4140950 0x7fe8e40b6220 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8c40682f0 0x7fe8c406a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e40b2e70 0x7fe8e4154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 --2- 172.21.15.144:0/3353299224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40cd8c0 0x7fe8e41419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 >> 172.21.15.144:0/3353299224 conn(0x7fe8e406aed0 msgr2=0x7fe8e4159f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 shutdown_connections 2023-11-28T11:52:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:35.995+0000 7fe8e9e07700 1 -- 172.21.15.144:0/3353299224 wait complete. 2023-11-28T11:52:36.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:52:36.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.130+0000 7f54aa3b1700 1 Processor -- start 2023-11-28T11:52:36.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.130+0000 7f54aa3b1700 1 -- start start 2023-11-28T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.130+0000 7f54aa3b1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f54a4071040 0x7f54a4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54aa3b1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f54a406dee0 0x7f54a406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54aa3b1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406cdc0 0x7f54a406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54aa3b1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a4071950 con 0x7f54a4071040 2023-11-28T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54aa3b1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a415d060 con 0x7f54a406dee0 2023-11-28T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54aa3b1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a415d340 con 0x7f54a406cdc0 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f54a4071040 0x7f54a4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f54a4071040 0x7f54a4071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36864/0 (socket says 172.21.15.144:36864) 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 -- 172.21.15.144:0/4077483239 learned_addr learned my addr 172.21.15.144:0/4077483239 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a894e700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406cdc0 0x7f54a406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a37fe700 1 --2- 172.21.15.144:0/4077483239 >> v2:172.21.15.177:3300/0 conn(0x7f54a406dee0 0x7f54a406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 -- 172.21.15.144:0/4077483239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406cdc0 msgr2=0x7f54a406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406cdc0 0x7f54a406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 -- 172.21.15.144:0/4077483239 >> v2:172.21.15.177:3300/0 conn(0x7f54a406dee0 msgr2=0x7f54a406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 --2- 172.21.15.144:0/4077483239 >> v2:172.21.15.177:3300/0 conn(0x7f54a406dee0 0x7f54a406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 -- 172.21.15.144:0/4077483239 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a415d620 con 0x7f54a4071040 2023-11-28T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a894e700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406cdc0 0x7f54a406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a37fe700 1 --2- 172.21.15.144:0/4077483239 >> v2:172.21.15.177:3300/0 conn(0x7f54a406dee0 0x7f54a406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a3fff700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4071410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f54a406fb00 tx=0x7f5494009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8807629a9999e73c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a2ffd700 1 -- 172.21.15.144:0/4077483239 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400b870 con 0x7f54a4071040 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a2ffd700 1 -- 172.21.15.144:0/4077483239 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549400b9d0 con 0x7f54a4071040 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.131+0000 7f54a2ffd700 1 -- 172.21.15.144:0/4077483239 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400bd60 con 0x7f54a4071040 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 -- 172.21.15.144:0/4077483239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 msgr2=0x7f54a4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4071410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f54a406fb00 tx=0x7f5494009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 -- 172.21.15.144:0/4077483239 shutdown_connections 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406cdc0 0x7f54a406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/4077483239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4071410 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/4077483239 >> v2:172.21.15.177:3300/0 conn(0x7f54a406dee0 0x7f54a406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 -- 172.21.15.144:0/4077483239 >> 172.21.15.144:0/4077483239 conn(0x7f54a40d0000 msgr2=0x7f54a40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:36.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 -- 172.21.15.144:0/4077483239 shutdown_connections 2023-11-28T11:52:36.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.132+0000 7f54aa3b1700 1 -- 172.21.15.144:0/4077483239 wait complete. 2023-11-28T11:52:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.133+0000 7f54aa3b1700 1 Processor -- start 2023-11-28T11:52:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.133+0000 7f54aa3b1700 1 -- start start 2023-11-28T11:52:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54aa3b1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54a406cdc0 0x7f54a415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54aa3b1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406dee0 0x7f54a41575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54aa3b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54aa3b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a415a080 con 0x7f54a4071040 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54aa3b1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4159120 con 0x7f54a406cdc0 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54aa3b1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a4159400 con 0x7f54a406dee0 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36874/0 (socket says 172.21.15.144:36874) 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 -- 172.21.15.144:0/3279253455 learned_addr learned my addr 172.21.15.144:0/3279253455 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a3fff700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54a406cdc0 0x7f54a415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a37fe700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406dee0 0x7f54a41575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 -- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406dee0 msgr2=0x7f54a41575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406dee0 0x7f54a41575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 -- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54a406cdc0 msgr2=0x7f54a415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54a406cdc0 0x7f54a415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.134+0000 7f54a894e700 1 -- 172.21.15.144:0/3279253455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5494009550 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54a37fe700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406dee0 0x7f54a41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54a894e700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4157b00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f549c007db0 tx=0x7f549c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00ddd0 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00f040 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a406e870 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c003c60 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.135+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a415d900 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.136+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a415ccd0 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.137+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f549c01c030 con 0x7f54a4071040 2023-11-28T11:52:36.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.137+0000 7f54a0ff9700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f54900682f0 0x7f549006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.137+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f549c0860e0 con 0x7f54a4071040 2023-11-28T11:52:36.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.137+0000 7f54a3fff700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f54900682f0 0x7f549006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.137+0000 7f54a3fff700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f54900682f0 0x7f549006a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5494009840 tx=0x7f5494008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.141+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f549c0528d0 con 0x7f54a4071040 2023-11-28T11:52:36.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.321+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54a415a1c0 con 0x7f54a4071040 2023-11-28T11:52:36.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.321+0000 7f54a0ff9700 1 -- 172.21.15.144:0/3279253455 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54a415a1c0 con 0x7f54a4071040 2023-11-28T11:52:36.322 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:52:36.324 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f54900682f0 msgr2=0x7f549006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f54900682f0 0x7f549006a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5494009840 tx=0x7f5494008030 comp rx=0 tx=0).stop 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 msgr2=0x7f54a4157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4157b00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f549c007db0 tx=0x7f549c00d3b0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 shutdown_connections 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54a4071040 0x7f54a4157b00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.323+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f54900682f0 0x7f549006a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.324+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54a406cdc0 0x7f54a415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.324+0000 7f54aa3b1700 1 --2- 172.21.15.144:0/3279253455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f54a406dee0 0x7f54a41575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.324+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 >> 172.21.15.144:0/3279253455 conn(0x7f54a40d0000 msgr2=0x7f54a40aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.324+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 shutdown_connections 2023-11-28T11:52:36.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.324+0000 7f54aa3b1700 1 -- 172.21.15.144:0/3279253455 wait complete. 2023-11-28T11:52:36.336 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:36.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.456+0000 7fcc1a441700 1 Processor -- start 2023-11-28T11:52:36.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.456+0000 7fcc1a441700 1 -- start start 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc1a441700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc14071040 0x7fcc14071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc1a441700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc1406dee0 0x7fcc1406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc1a441700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc14071950 con 0x7fcc1406cdc0 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc1a441700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc14161550 con 0x7fcc1406dee0 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc1a441700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1415d060 con 0x7fcc14071040 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc189de700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcc1406cdc0 0x7fcc1406d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53594/0 (socket says 172.21.15.144:53594) 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc189de700 1 -- 172.21.15.144:0/669094161 learned_addr learned my addr 172.21.15.144:0/669094161 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc137fe700 1 --2- 172.21.15.144:0/669094161 >> v2:172.21.15.177:3300/0 conn(0x7fcc1406dee0 0x7fcc1406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc13fff700 1 --2- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc14071040 0x7fcc14071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc12ffd700 1 -- 172.21.15.144:0/669094161 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580268658 0 0) 0x7fcc14071950 con 0x7fcc1406cdc0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.457+0000 7fcc12ffd700 1 -- 172.21.15.144:0/669094161 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc0035b0 con 0x7fcc1406cdc0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc137fe700 1 -- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc14071040 msgr2=0x7fcc14071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc137fe700 1 --2- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc14071040 0x7fcc14071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc137fe700 1 -- 172.21.15.144:0/669094161 >> v1:172.21.15.144:6789/0 conn(0x7fcc1406cdc0 legacy=0x7fcc1406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc137fe700 1 -- 172.21.15.144:0/669094161 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1415d340 con 0x7fcc1406dee0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc137fe700 1 --2- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc1406e350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009840 tx=0x7fcc04009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df4e9d97c7645c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc12ffd700 1 -- 172.21.15.144:0/669094161 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04011830 con 0x7fcc1406dee0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc12ffd700 1 -- 172.21.15.144:0/669094161 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04011990 con 0x7fcc1406dee0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.458+0000 7fcc12ffd700 1 -- 172.21.15.144:0/669094161 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04011d20 con 0x7fcc1406dee0 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 -- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 msgr2=0x7fcc1406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 --2- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc1406e350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009840 tx=0x7fcc04009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:36.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 -- 172.21.15.144:0/669094161 shutdown_connections 2023-11-28T11:52:36.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 --2- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc14071040 0x7fcc14071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 --2- 172.21.15.144:0/669094161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc1406e350 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 -- 172.21.15.144:0/669094161 >> 172.21.15.144:0/669094161 conn(0x7fcc140d0000 msgr2=0x7fcc140d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:36.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 -- 172.21.15.144:0/669094161 shutdown_connections 2023-11-28T11:52:36.462 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.459+0000 7fcc1a441700 1 -- 172.21.15.144:0/669094161 wait complete. 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.460+0000 7fcc1a441700 1 Processor -- start 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.460+0000 7fcc1a441700 1 -- start start 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc1406cdc0 0x7fcc140b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc140b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 0x7fcc14155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14155ff0 con 0x7fcc14071040 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140c4b70 con 0x7fcc1406dee0 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140c6b20 con 0x7fcc1406cdc0 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 0x7fcc14155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 0x7fcc14155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36888/0 (socket says 172.21.15.144:36888) 2023-11-28T11:52:36.465 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 -- 172.21.15.144:0/3843054901 learned_addr learned my addr 172.21.15.144:0/3843054901 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc137fe700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc140b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc13fff700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc1406cdc0 0x7fcc140b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 -- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc1406cdc0 msgr2=0x7fcc140b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc1406cdc0 0x7fcc140b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 -- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 msgr2=0x7fcc140b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc140b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 -- 172.21.15.144:0/3843054901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040094a0 con 0x7fcc14071040 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc137fe700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc140b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc189de700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 0x7fcc14155660 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00cad0 tx=0x7fcc0c00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.466 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00aac0 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c00ac20 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1415d900 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.461+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c012400 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.462+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1415d620 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.462+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1406e890 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.463+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc0c019030 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.463+0000 7fcc10ff9700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc000682f0 0x7fcc0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.463+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc0c085930 con 0x7fcc14071040 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.463+0000 7fcc13fff700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc000682f0 0x7fcc0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.464+0000 7fcc13fff700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc000682f0 0x7fcc0006a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009ad0 tx=0x7fcc08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.468+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcc0c052110 con 0x7fcc14071040 2023-11-28T11:52:36.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.692+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc1406b040 con 0x7fcc14071040 2023-11-28T11:52:36.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.693+0000 7fcc10ff9700 1 -- 172.21.15.144:0/3843054901 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fcc0c055730 con 0x7fcc14071040 2023-11-28T11:52:36.694 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:36.694 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:36.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc000682f0 msgr2=0x7fcc0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc000682f0 0x7fcc0006a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009ad0 tx=0x7fcc08008040 comp rx=0 tx=0).stop 2023-11-28T11:52:36.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 msgr2=0x7fcc14155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 0x7fcc14155660 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00cad0 tx=0x7fcc0c00ce90 comp rx=0 tx=0).stop 2023-11-28T11:52:36.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 shutdown_connections 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14071040 0x7fcc14155660 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc000682f0 0x7fcc0006a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc1406cdc0 0x7fcc140b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 --2- 172.21.15.144:0/3843054901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc1406dee0 0x7fcc140b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 >> 172.21.15.144:0/3843054901 conn(0x7fcc140d0000 msgr2=0x7fcc140cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 shutdown_connections 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.695+0000 7fcc1a441700 1 -- 172.21.15.144:0/3843054901 wait complete. 2023-11-28T11:52:36.697 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:36.708 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:36.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.827+0000 7f48261f2700 1 Processor -- start 2023-11-28T11:52:36.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.828+0000 7f48261f2700 1 -- start start 2023-11-28T11:52:36.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f48261f2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4820140950 0x7f48200b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f48261f2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f48200b2e70 0x7f482006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f48261f2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482013ff00 con 0x7f48200cd8c0 2023-11-28T11:52:36.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f48261f2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48201404c0 con 0x7f4820140950 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f48261f2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4820140dc0 con 0x7f48200b2e70 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4820140950 0x7f48200b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4820140950 0x7f48200b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36894/0 (socket says 172.21.15.144:36894) 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 -- 172.21.15.144:0/2414178886 learned_addr learned my addr 172.21.15.144:0/2414178886 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481ffff700 1 --2- 172.21.15.144:0/2414178886 >> v2:172.21.15.177:3300/0 conn(0x7f48200b2e70 0x7f482006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 -- 172.21.15.144:0/2414178886 >> v2:172.21.15.177:3300/0 conn(0x7f48200b2e70 msgr2=0x7f482006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 --2- 172.21.15.144:0/2414178886 >> v2:172.21.15.177:3300/0 conn(0x7f48200b2e70 0x7f482006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 -- 172.21.15.144:0/2414178886 >> v1:172.21.15.177:6789/0 conn(0x7f48200cd8c0 legacy=0x7f48200cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 -- 172.21.15.144:0/2414178886 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482015aa80 con 0x7f4820140950 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481ffff700 1 --2- 172.21.15.144:0/2414178886 >> v2:172.21.15.177:3300/0 conn(0x7f48200b2e70 0x7f482006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481effd700 1 --2- 172.21.15.144:0/2414178886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4820140950 0x7f48200b2930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4820144350 tx=0x7f4810009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409dacd35d0536de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481e7fc700 1 -- 172.21.15.144:0/2414178886 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000b870 con 0x7f4820140950 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481e7fc700 1 -- 172.21.15.144:0/2414178886 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481000b9d0 con 0x7f4820140950 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.829+0000 7f481e7fc700 1 -- 172.21.15.144:0/2414178886 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000bd60 con 0x7f4820140950 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 -- 172.21.15.144:0/2414178886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4820140950 msgr2=0x7f48200b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 --2- 172.21.15.144:0/2414178886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4820140950 0x7f48200b2930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4820144350 tx=0x7f4810009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 -- 172.21.15.144:0/2414178886 shutdown_connections 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 --2- 172.21.15.144:0/2414178886 >> v2:172.21.15.177:3300/0 conn(0x7f48200b2e70 0x7f482006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 --2- 172.21.15.144:0/2414178886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4820140950 0x7f48200b2930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 -- 172.21.15.144:0/2414178886 >> 172.21.15.144:0/2414178886 conn(0x7f482006aed0 msgr2=0x7f482006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 -- 172.21.15.144:0/2414178886 shutdown_connections 2023-11-28T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.830+0000 7f48261f2700 1 -- 172.21.15.144:0/2414178886 wait complete. 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.831+0000 7f48261f2700 1 Processor -- start 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.831+0000 7f48261f2700 1 -- start start 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f48261f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 0x7f4820153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f48261f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48200cd8c0 0x7f48200c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f48261f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4820140950 0x7f48200cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f48261f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c8400 con 0x7f48200b2e70 2023-11-28T11:52:36.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f48261f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c5760 con 0x7f48200cd8c0 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f48261f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200c5a40 con 0x7f4820140950 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 0x7f4820153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 0x7f4820153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36898/0 (socket says 172.21.15.144:36898) 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 -- 172.21.15.144:0/1108970846 learned_addr learned my addr 172.21.15.144:0/1108970846 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481effd700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48200cd8c0 0x7f48200c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481ffff700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4820140950 0x7f48200cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 -- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4820140950 msgr2=0x7f48200cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4820140950 0x7f48200cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 -- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48200cd8c0 msgr2=0x7f48200c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48200cd8c0 0x7f48200c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481f7fe700 1 -- 172.21.15.144:0/1108970846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009550 con 0x7f48200b2e70 2023-11-28T11:52:36.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481effd700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48200cd8c0 0x7f48200c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.832+0000 7f481ffff700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4820140950 0x7f48200cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.833+0000 7f481f7fe700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 0x7f4820153a30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f481400c4d0 tx=0x7f481400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.833+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140052d0 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.833+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4814005430 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.833+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48200b35f0 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.833+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140038f0 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.833+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48200b3310 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.834+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820140150 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.834+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4814019030 con 0x7f48200b2e70 2023-11-28T11:52:36.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.835+0000 7f4807fff700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f480c0682f0 0x7f480c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:36.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.835+0000 7f481effd700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f480c0682f0 0x7f480c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:36.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.835+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4814085090 con 0x7f48200b2e70 2023-11-28T11:52:36.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.835+0000 7f481effd700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f480c0682f0 0x7f480c06a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4820144350 tx=0x7f4810008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:36.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:36.839+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4814051890 con 0x7f48200b2e70 2023-11-28T11:52:37.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.023+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4820068690 con 0x7f48200b2e70 2023-11-28T11:52:37.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.024+0000 7f4807fff700 1 -- 172.21.15.144:0/1108970846 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f4814054eb0 con 0x7f48200b2e70 2023-11-28T11:52:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:37.025 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:10.310687+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f480c0682f0 msgr2=0x7f480c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f480c0682f0 0x7f480c06a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4820144350 tx=0x7f4810008030 comp rx=0 tx=0).stop 2023-11-28T11:52:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 msgr2=0x7f4820153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 0x7f4820153a30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f481400c4d0 tx=0x7f481400c890 comp rx=0 tx=0).stop 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 shutdown_connections 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48200b2e70 0x7f4820153a30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f480c0682f0 0x7f480c06a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4820140950 0x7f48200cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 --2- 172.21.15.144:0/1108970846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48200cd8c0 0x7f48200c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 >> 172.21.15.144:0/1108970846 conn(0x7f482006aed0 msgr2=0x7f48200d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 shutdown_connections 2023-11-28T11:52:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.026+0000 7f48261f2700 1 -- 172.21.15.144:0/1108970846 wait complete. 2023-11-28T11:52:37.039 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:52:37.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:37.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.158+0000 7fcdeed19700 1 Processor -- start 2023-11-28T11:52:37.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.158+0000 7fcdeed19700 1 -- start start 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcdeed19700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde8071040 0x7fcde8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcdeed19700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcde806dee0 0x7fcde806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcdeed19700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde8071950 con 0x7fcde806cdc0 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcdeed19700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde8161550 con 0x7fcde806dee0 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcdeed19700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde815d060 con 0x7fcde8071040 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcde7fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcde806dee0 0x7fcde806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcdecab5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde8071040 0x7fcde8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcded2b6700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fcde806cdc0 0x7fcde806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:54110/0 (socket says 172.21.15.144:54110) 2023-11-28T11:52:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.159+0000 7fcded2b6700 1 -- 172.21.15.144:0/1044701911 learned_addr learned my addr 172.21.15.144:0/1044701911 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcdecab5700 1 -- 172.21.15.144:0/1044701911 >> v2:172.21.15.177:3300/0 conn(0x7fcde806dee0 msgr2=0x7fcde806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcdecab5700 1 --2- 172.21.15.144:0/1044701911 >> v2:172.21.15.177:3300/0 conn(0x7fcde806dee0 0x7fcde806e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcdecab5700 1 -- 172.21.15.144:0/1044701911 >> v1:172.21.15.177:6789/0 conn(0x7fcde806cdc0 legacy=0x7fcde806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcdecab5700 1 -- 172.21.15.144:0/1044701911 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde815d340 con 0x7fcde8071040 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcde77fe700 1 -- 172.21.15.144:0/1044701911 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323716479 0 0) 0x7fcde8071950 con 0x7fcde806cdc0 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcde7fff700 1 --2- 172.21.15.144:0/1044701911 >> v2:172.21.15.177:3300/0 conn(0x7fcde806dee0 0x7fcde806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcdecab5700 1 --2- 172.21.15.144:0/1044701911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde8071040 0x7fcde8071410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009840 tx=0x7fcddc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c78e14c963df34e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcde77fe700 1 -- 172.21.15.144:0/1044701911 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc011830 con 0x7fcde8071040 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcde77fe700 1 -- 172.21.15.144:0/1044701911 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc011990 con 0x7fcde8071040 2023-11-28T11:52:37.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.160+0000 7fcde77fe700 1 -- 172.21.15.144:0/1044701911 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc011d20 con 0x7fcde8071040 2023-11-28T11:52:37.165 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 -- 172.21.15.144:0/1044701911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde8071040 msgr2=0x7fcde8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.165 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1044701911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde8071040 0x7fcde8071410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009840 tx=0x7fcddc009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:37.165 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 -- 172.21.15.144:0/1044701911 shutdown_connections 2023-11-28T11:52:37.165 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1044701911 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde8071040 0x7fcde8071410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.165 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1044701911 >> v2:172.21.15.177:3300/0 conn(0x7fcde806dee0 0x7fcde806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 -- 172.21.15.144:0/1044701911 >> 172.21.15.144:0/1044701911 conn(0x7fcde80d0000 msgr2=0x7fcde80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 -- 172.21.15.144:0/1044701911 shutdown_connections 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.161+0000 7fcdeed19700 1 -- 172.21.15.144:0/1044701911 wait complete. 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.162+0000 7fcdeed19700 1 Processor -- start 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.162+0000 7fcdeed19700 1 -- start start 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdeed19700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde806cdc0 0x7fcde80b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdeed19700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 0x7fcde814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdeed19700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8071040 0x7fcde80c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdeed19700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80c4a30 con 0x7fcde806dee0 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdeed19700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde814e870 con 0x7fcde8071040 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdeed19700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80c3e20 con 0x7fcde806cdc0 2023-11-28T11:52:37.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 0x7fcde814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 0x7fcde814e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36914/0 (socket says 172.21.15.144:36914) 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 -- 172.21.15.144:0/1701034612 learned_addr learned my addr 172.21.15.144:0/1701034612 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcdecab5700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde806cdc0 0x7fcde80b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcded2b6700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8071040 0x7fcde80c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 -- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde806cdc0 msgr2=0x7fcde80b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde806cdc0 0x7fcde80b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 -- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8071040 msgr2=0x7fcde80c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8071040 0x7fcde80c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcde7fff700 1 -- 172.21.15.144:0/1701034612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40094a0 con 0x7fcde806dee0 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.163+0000 7fcded2b6700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8071040 0x7fcde80c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:37.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.164+0000 7fcde7fff700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 0x7fcde814e330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400ebe0 tx=0x7fcdd400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.164+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400c850 con 0x7fcde806dee0 2023-11-28T11:52:37.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.164+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd400c9b0 con 0x7fcde806dee0 2023-11-28T11:52:37.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.164+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc0094a0 con 0x7fcde806dee0 2023-11-28T11:52:37.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.164+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400cd40 con 0x7fcde806dee0 2023-11-28T11:52:37.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.164+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde815d900 con 0x7fcde806dee0 2023-11-28T11:52:37.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.165+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde815d620 con 0x7fcde806dee0 2023-11-28T11:52:37.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.165+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcdd4019030 con 0x7fcde806dee0 2023-11-28T11:52:37.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.166+0000 7fcde57fa700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcdcc0682f0 0x7fcdcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.166+0000 7fcdecab5700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcdcc0682f0 0x7fcdcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.166+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcdd4085cb0 con 0x7fcde806dee0 2023-11-28T11:52:37.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.166+0000 7fcdecab5700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcdcc0682f0 0x7fcdcc06a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000c00 tx=0x7fcddc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.170+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcdd4052420 con 0x7fcde806dee0 2023-11-28T11:52:37.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.396+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcde806e890 con 0x7fcde806dee0 2023-11-28T11:52:37.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.396+0000 7fcde57fa700 1 -- 172.21.15.144:0/1701034612 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7fcdd400cea0 con 0x7fcde806dee0 2023-11-28T11:52:37.397 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:37.397 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:37.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcdcc0682f0 msgr2=0x7fcdcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcdcc0682f0 0x7fcdcc06a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcddc000c00 tx=0x7fcddc008030 comp rx=0 tx=0).stop 2023-11-28T11:52:37.399 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 msgr2=0x7fcde814e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 0x7fcde814e330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400ebe0 tx=0x7fcdd400eef0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 shutdown_connections 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcdcc0682f0 0x7fcdcc06a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8071040 0x7fcde80c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcde806cdc0 0x7fcde80b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 --2- 172.21.15.144:0/1701034612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde806dee0 0x7fcde814e330 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.398+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 >> 172.21.15.144:0/1701034612 conn(0x7fcde80d0000 msgr2=0x7fcde806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.399+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 shutdown_connections 2023-11-28T11:52:37.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.399+0000 7fcdeed19700 1 -- 172.21.15.144:0/1701034612 wait complete. 2023-11-28T11:52:37.401 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:37.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:37.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.530+0000 7f1631cae700 1 Processor -- start 2023-11-28T11:52:37.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.530+0000 7f1631cae700 1 -- start start 2023-11-28T11:52:37.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f1631cae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f162c071040 0x7f162c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f1631cae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f162c06dee0 0x7f162c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f1631cae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c071950 con 0x7f162c06cdc0 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f1631cae700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c161550 con 0x7f162c071040 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f1631cae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c15d060 con 0x7f162c06dee0 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f162b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f162c071040 0x7f162c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f162bfff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f162c06cdc0 0x7f162c06d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53634/0 (socket says 172.21.15.144:53634) 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f162bfff700 1 -- 172.21.15.144:0/3802229098 learned_addr learned my addr 172.21.15.144:0/3802229098 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.531+0000 7f162affd700 1 --2- 172.21.15.144:0/3802229098 >> v2:172.21.15.177:3300/0 conn(0x7f162c06dee0 0x7f162c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162b7fe700 1 -- 172.21.15.144:0/3802229098 >> v2:172.21.15.177:3300/0 conn(0x7f162c06dee0 msgr2=0x7f162c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162b7fe700 1 --2- 172.21.15.144:0/3802229098 >> v2:172.21.15.177:3300/0 conn(0x7f162c06dee0 0x7f162c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162b7fe700 1 -- 172.21.15.144:0/3802229098 >> v1:172.21.15.144:6789/0 conn(0x7f162c06cdc0 legacy=0x7f162c06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162b7fe700 1 -- 172.21.15.144:0/3802229098 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c15d340 con 0x7f162c071040 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162b7fe700 1 --2- 172.21.15.144:0/3802229098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c071410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f162c06fb00 tx=0x7f1620009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412985a49565187d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162a7fc700 1 -- 172.21.15.144:0/3802229098 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620011830 con 0x7f162c071040 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162a7fc700 1 -- 172.21.15.144:0/3802229098 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620011990 con 0x7f162c071040 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f162a7fc700 1 -- 172.21.15.144:0/3802229098 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620011d20 con 0x7f162c071040 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f1631cae700 1 -- 172.21.15.144:0/3802229098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 msgr2=0x7f162c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f1631cae700 1 --2- 172.21.15.144:0/3802229098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c071410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f162c06fb00 tx=0x7f1620009b20 comp rx=0 tx=0).stop 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f1631cae700 1 -- 172.21.15.144:0/3802229098 shutdown_connections 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f1631cae700 1 --2- 172.21.15.144:0/3802229098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c071410 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f1631cae700 1 --2- 172.21.15.144:0/3802229098 >> v2:172.21.15.177:3300/0 conn(0x7f162c06dee0 0x7f162c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.532+0000 7f1631cae700 1 -- 172.21.15.144:0/3802229098 >> 172.21.15.144:0/3802229098 conn(0x7f162c0d0000 msgr2=0x7f162c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.533+0000 7f1631cae700 1 -- 172.21.15.144:0/3802229098 shutdown_connections 2023-11-28T11:52:37.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.533+0000 7f1631cae700 1 -- 172.21.15.144:0/3802229098 wait complete. 2023-11-28T11:52:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.534+0000 7f1631cae700 1 Processor -- start 2023-11-28T11:52:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.534+0000 7f1631cae700 1 -- start start 2023-11-28T11:52:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f1631cae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c06cdc0 0x7f162c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f1631cae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c06dee0 0x7f162c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f1631cae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f1631cae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0ad640 con 0x7f162c071040 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f1631cae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0ad920 con 0x7f162c06dee0 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f1631cae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0adc00 con 0x7f162c06cdc0 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f162bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f162bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36926/0 (socket says 172.21.15.144:36926) 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f162bfff700 1 -- 172.21.15.144:0/1355490130 learned_addr learned my addr 172.21.15.144:0/1355490130 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f162affd700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c06dee0 0x7f162c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.535+0000 7f162b7fe700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c06cdc0 0x7f162c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f162bfff700 1 -- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c06cdc0 msgr2=0x7f162c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f162bfff700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c06cdc0 0x7f162c0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f162bfff700 1 -- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c06dee0 msgr2=0x7f162c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f162bfff700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c06dee0 0x7f162c0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f162bfff700 1 -- 172.21.15.144:0/1355490130 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200094a0 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f162bfff700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c0b30a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f162400ca80 tx=0x7f162400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162400a9e0 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162400ab40 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c15d900 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624012400 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.536+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c15d620 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.537+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c06e870 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.537+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1624019030 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.538+0000 7f1613fff700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16180682f0 0x7f161806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.538+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1624085580 con 0x7f162c071040 2023-11-28T11:52:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.538+0000 7f162b7fe700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16180682f0 0x7f161806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.538+0000 7f162b7fe700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16180682f0 0x7f161806a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f16200090b0 tx=0x7f162001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.542+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f162404b5e0 con 0x7f162c071040 2023-11-28T11:52:37.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.768+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f162c06f4a0 con 0x7f162c071040 2023-11-28T11:52:37.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.768+0000 7f1613fff700 1 -- 172.21.15.144:0/1355490130 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4657 (secure 0 0 0) 0x7f1624051e90 con 0x7f162c071040 2023-11-28T11:52:37.769 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:37.769 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:09.307915+0000","modified":"2023-11-28T11:52:12.326845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6836/2826166019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2826166019},{"type":"v1","addr":"172.21.15.144:6836","nonce":2826166019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:52:37.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16180682f0 msgr2=0x7f161806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16180682f0 0x7f161806a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f16200090b0 tx=0x7f162001c040 comp rx=0 tx=0).stop 2023-11-28T11:52:37.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 msgr2=0x7f162c0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c0b30a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f162400ca80 tx=0x7f162400cd90 comp rx=0 tx=0).stop 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 shutdown_connections 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f162c071040 0x7f162c0b30a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16180682f0 0x7f161806a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f162c06cdc0 0x7f162c0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 --2- 172.21.15.144:0/1355490130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f162c06dee0 0x7f162c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 >> 172.21.15.144:0/1355490130 conn(0x7f162c0d0000 msgr2=0x7f162c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.770+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 shutdown_connections 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.771+0000 7f1631cae700 1 -- 172.21.15.144:0/1355490130 wait complete. 2023-11-28T11:52:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 16 2023-11-28T11:52:37.783 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:37.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.902+0000 7fc0d0c0e700 1 Processor -- start 2023-11-28T11:52:37.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.902+0000 7fc0d0c0e700 1 -- start start 2023-11-28T11:52:37.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0d0c0e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0cc160740 0x7fc0cc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0d0c0e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0cc160b10 0x7fc0cc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0d0c0e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0cc06bbb0 con 0x7fc0cc06b060 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0d0c0e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc0ca720 con 0x7fc0cc160b10 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0d0c0e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc0cc030 con 0x7fc0cc160740 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0c9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0cc160b10 0x7fc0cc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0cad9d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc0cc06b060 0x7fc0cc06b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53644/0 (socket says 172.21.15.144:53644) 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0c9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0cc160b10 0x7fc0cc0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36932/0 (socket says 172.21.15.144:36932) 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.903+0000 7fc0cad9d700 1 -- 172.21.15.144:0/3735042883 learned_addr learned my addr 172.21.15.144:0/3735042883 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0ca59c700 1 --2- 172.21.15.144:0/3735042883 >> v2:172.21.15.177:3300/0 conn(0x7fc0cc160740 0x7fc0cc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c9d9b700 1 -- 172.21.15.144:0/3735042883 >> v2:172.21.15.177:3300/0 conn(0x7fc0cc160740 msgr2=0x7fc0cc0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:37.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c9d9b700 1 --2- 172.21.15.144:0/3735042883 >> v2:172.21.15.177:3300/0 conn(0x7fc0cc160740 0x7fc0cc0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c9d9b700 1 -- 172.21.15.144:0/3735042883 >> v1:172.21.15.144:6789/0 conn(0x7fc0cc06b060 legacy=0x7fc0cc06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c9d9b700 1 -- 172.21.15.144:0/3735042883 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc0c9e10 con 0x7fc0cc160b10 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c959a700 1 -- 172.21.15.144:0/3735042883 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354587310 0 0) 0x7fc0cc06bbb0 con 0x7fc0cc06b060 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0ca59c700 1 --2- 172.21.15.144:0/3735042883 >> v2:172.21.15.177:3300/0 conn(0x7fc0cc160740 0x7fc0cc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c9d9b700 1 --2- 172.21.15.144:0/3735042883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc0cd390 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0b23b0 tx=0x7fc0bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e118575d85c1d28e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c959a700 1 -- 172.21.15.144:0/3735042883 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc011830 con 0x7fc0cc160b10 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c959a700 1 -- 172.21.15.144:0/3735042883 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc011990 con 0x7fc0cc160b10 2023-11-28T11:52:37.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.904+0000 7fc0c959a700 1 -- 172.21.15.144:0/3735042883 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc011d20 con 0x7fc0cc160b10 2023-11-28T11:52:37.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/3735042883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 msgr2=0x7fc0cc0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/3735042883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc0cd390 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0b23b0 tx=0x7fc0bc009b20 comp rx=0 tx=0).stop 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/3735042883 shutdown_connections 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/3735042883 >> v2:172.21.15.177:3300/0 conn(0x7fc0cc160740 0x7fc0cc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/3735042883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc0cd390 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/3735042883 >> 172.21.15.144:0/3735042883 conn(0x7fc0cc0b12b0 msgr2=0x7fc0cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/3735042883 shutdown_connections 2023-11-28T11:52:37.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.905+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/3735042883 wait complete. 2023-11-28T11:52:37.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.906+0000 7fc0d0c0e700 1 Processor -- start 2023-11-28T11:52:37.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.906+0000 7fc0d0c0e700 1 -- start start 2023-11-28T11:52:37.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0d0c0e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc06b060 0x7fc0cc0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0d0c0e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc160740 0x7fc0cc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0d0c0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc15af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0d0c0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15fd40 con 0x7fc0cc160b10 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0d0c0e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15ddf0 con 0x7fc0cc06b060 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0d0c0e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15f780 con 0x7fc0cc160740 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc15af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36938/0 (socket says 172.21.15.144:36938) 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 -- 172.21.15.144:0/1047647455 learned_addr learned my addr 172.21.15.144:0/1047647455 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0c9d9b700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc160740 0x7fc0cc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0ca59c700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc06b060 0x7fc0cc0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 -- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc160740 msgr2=0x7fc0cc15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc160740 0x7fc0cc15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 -- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc06b060 msgr2=0x7fc0cc0c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc06b060 0x7fc0cc0c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 -- 172.21.15.144:0/1047647455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0094a0 con 0x7fc0cc160b10 2023-11-28T11:52:37.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0ca59c700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc06b060 0x7fc0cc0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.907+0000 7fc0cad9d700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc15af40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400cba0 tx=0x7fc0c400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.908+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400aad0 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.908+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400ac50 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.908+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc0ca0d0 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.908+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40070d0 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.908+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc06efc0 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.909+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc0ca3b0 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.909+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc0c401a030 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.909+0000 7fc0baffd700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0ac0682f0 0x7fc0ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.910+0000 7fc0ca59c700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0ac0682f0 0x7fc0ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.910+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc0c40858f0 con 0x7fc0cc160b10 2023-11-28T11:52:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.910+0000 7fc0ca59c700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0ac0682f0 0x7fc0ac06a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0006fd0 tx=0x7fc0c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:37.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:37.914+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc0c4052150 con 0x7fc0cc160b10 2023-11-28T11:52:38.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.098+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0cc066110 con 0x7fc0cc160b10 2023-11-28T11:52:38.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.099+0000 7fc0baffd700 1 -- 172.21.15.144:0/1047647455 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17969 (secure 0 0 0) 0x7fc0c4018090 con 0x7fc0cc160b10 2023-11-28T11:52:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:10.310687+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:07.570239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:08.264687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0ac0682f0 msgr2=0x7fc0ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0ac0682f0 0x7fc0ac06a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0006fd0 tx=0x7fc0c0008040 comp rx=0 tx=0).stop 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 msgr2=0x7fc0cc15af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc15af40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400cba0 tx=0x7fc0c400ceb0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 shutdown_connections 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0cc160b10 0x7fc0cc15af40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc0ac0682f0 0x7fc0ac06a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0cc06b060 0x7fc0cc0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 --2- 172.21.15.144:0/1047647455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0cc160740 0x7fc0cc15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 >> 172.21.15.144:0/1047647455 conn(0x7fc0cc0b12b0 msgr2=0x7fc0cc06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 shutdown_connections 2023-11-28T11:52:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.101+0000 7fc0d0c0e700 1 -- 172.21.15.144:0/1047647455 wait complete. 2023-11-28T11:52:38.114 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:52:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.234+0000 7f9c62985700 1 Processor -- start 2023-11-28T11:52:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.234+0000 7f9c62985700 1 -- start start 2023-11-28T11:52:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c62985700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c5c0d0a00 0x7f9c5c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c62985700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0b2930 0x7f9c5c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c62985700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c06b420 con 0x7f9c5c0ce040 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c62985700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b32f0 con 0x7f9c5c0d0a00 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c62985700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c06ff60 con 0x7f9c5c0b2930 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c5c0d0a00 0x7f9c5c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c5c0d0a00 0x7f9c5c15df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36944/0 (socket says 172.21.15.144:36944) 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/66324349 learned_addr learned my addr 172.21.15.144:0/66324349 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c60f22700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0b2930 0x7f9c5c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/66324349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0b2930 msgr2=0x7f9c5c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0b2930 0x7f9c5c0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/66324349 >> v1:172.21.15.177:6789/0 conn(0x7f9c5c0ce040 legacy=0x7f9c5c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:38.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/66324349 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c06b700 con 0x7f9c5c0d0a00 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c60f22700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0b2930 0x7f9c5c0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.235+0000 7f9c5b7fe700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0d0a00 0x7f9c5c15df10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098f0 tx=0x7f9c50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f171d6c98bd4fc8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c5affd700 1 -- 172.21.15.144:0/66324349 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004230 con 0x7f9c5c0d0a00 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c5affd700 1 -- 172.21.15.144:0/66324349 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50004390 con 0x7f9c5c0d0a00 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c5affd700 1 -- 172.21.15.144:0/66324349 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500036a0 con 0x7f9c5c0d0a00 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 -- 172.21.15.144:0/66324349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0d0a00 msgr2=0x7f9c5c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0d0a00 0x7f9c5c15df10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098f0 tx=0x7f9c50009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 -- 172.21.15.144:0/66324349 shutdown_connections 2023-11-28T11:52:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0b2930 0x7f9c5c0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 --2- 172.21.15.144:0/66324349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0d0a00 0x7f9c5c15df10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 -- 172.21.15.144:0/66324349 >> 172.21.15.144:0/66324349 conn(0x7f9c5c06aed0 msgr2=0x7f9c5c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:38.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 -- 172.21.15.144:0/66324349 shutdown_connections 2023-11-28T11:52:38.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.236+0000 7f9c62985700 1 -- 172.21.15.144:0/66324349 wait complete. 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.237+0000 7f9c62985700 1 Processor -- start 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 -- start start 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c5c0b2930 0x7f9c5c14a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 0x7f9c5c0c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0d0a00 0x7f9c5c0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c14a6f0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c6950 con 0x7f9c5c0b2930 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c62985700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c5d40 con 0x7f9c5c0d0a00 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c5b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 0x7f9c5c0c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c5b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 0x7f9c5c0c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36952/0 (socket says 172.21.15.144:36952) 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/2133067909 learned_addr learned my addr 172.21.15.144:0/2133067909 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c5bfff700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c5c0b2930 0x7f9c5c14a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.238+0000 7f9c60f22700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0d0a00 0x7f9c5c0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0d0a00 msgr2=0x7f9c5c0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c5b7fe700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0d0a00 0x7f9c5c0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c5c0b2930 msgr2=0x7f9c5c14a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c5b7fe700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c5c0b2930 0x7f9c5c14a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c5b7fe700 1 -- 172.21.15.144:0/2133067909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50009550 con 0x7f9c5c0ce040 2023-11-28T11:52:38.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c5b7fe700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 0x7f9c5c0c53d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000b660 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5000b7c0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c06bcc0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000bb70 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.239+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c06b9e0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.240+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c06bfa0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.240+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c50017020 con 0x7f9c5c0ce040 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.241+0000 7f9c58ff9700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c480682f0 0x7f9c4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.241+0000 7f9c5bfff700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c480682f0 0x7f9c4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.241+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c500875e0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.241+0000 7f9c5bfff700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c480682f0 0x7f9c4806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006fd0 tx=0x7f9c4c0090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:38.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.245+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c50053e40 con 0x7f9c5c0ce040 2023-11-28T11:52:38.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.466+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9c5c0c6ac0 con 0x7f9c5c0ce040 2023-11-28T11:52:38.756 INFO:tasks.ceph.mon.a.smithi144.stderr:2023-11-28T11:52:38.755+0000 7fdb2a043700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:52:38.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.762+0000 7f9c58ff9700 1 -- 172.21.15.144:0/2133067909 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9c50057460 con 0x7f9c5c0ce040 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c480682f0 msgr2=0x7f9c4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c480682f0 0x7f9c4806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006fd0 tx=0x7f9c4c0090e0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 msgr2=0x7f9c5c0c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 0x7f9c5c0c53d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000c00 tx=0x7f9c50004750 comp rx=0 tx=0).stop 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 shutdown_connections 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c480682f0 0x7f9c4806a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c5c0d0a00 0x7f9c5c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c5c0b2930 0x7f9c5c14a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 --2- 172.21.15.144:0/2133067909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c5c0ce040 0x7f9c5c0c53d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 >> 172.21.15.144:0/2133067909 conn(0x7f9c5c06aed0 msgr2=0x7f9c5c06e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:38.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 shutdown_connections 2023-11-28T11:52:38.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.765+0000 7f9c62985700 1 -- 172.21.15.144:0/2133067909 wait complete. 2023-11-28T11:52:38.768 INFO:teuthology.orchestra.run.smithi144.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:52:38.768 INFO:tasks.ceph.mds.a.smithi144.stderr: -7315> 2023-11-28T11:52:12.806+0000 7ff8c3457700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:52:38.784 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:52:38.809 INFO:tasks.ceph.mds.a.smithi144.stdout:starting mds.a at 2023-11-28T11:52:38.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.910+0000 7fc51cd5d700 1 Processor -- start 2023-11-28T11:52:38.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.910+0000 7fc51cd5d700 1 -- start start 2023-11-28T11:52:38.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51cd5d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5180cd8c0 0x7fc518140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51cd5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc51806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51cd5d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51813ff00 con 0x7fc5180cdc90 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51cd5d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5181404c0 con 0x7fc5180cd8c0 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51cd5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51815aa80 con 0x7fc51806f850 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51659c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5180cd8c0 0x7fc518140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc515d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc5180cdc90 0x7fc5180b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53680/0 (socket says 172.21.15.144:53680) 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc515d9b700 1 -- 172.21.15.144:0/2553479433 learned_addr learned my addr 172.21.15.144:0/2553479433 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc516d9d700 1 --2- 172.21.15.144:0/2553479433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc51806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51659c700 1 -- 172.21.15.144:0/2553479433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 msgr2=0x7fc51806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:38.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51659c700 1 --2- 172.21.15.144:0/2553479433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc51806fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51659c700 1 -- 172.21.15.144:0/2553479433 >> v1:172.21.15.144:6789/0 conn(0x7fc5180cdc90 legacy=0x7fc5180b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51659c700 1 -- 172.21.15.144:0/2553479433 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180b3040 con 0x7fc5180cd8c0 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.911+0000 7fc51559a700 1 -- 172.21.15.144:0/2553479433 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186030654 0 0) 0x7fc51813ff00 con 0x7fc5180cdc90 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51659c700 1 --2- 172.21.15.144:0/2553479433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc518140950 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009820 tx=0x7fc50c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a612e03aaa3577f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51559a700 1 -- 172.21.15.144:0/2553479433 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c005e30 con 0x7fc5180cd8c0 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51559a700 1 -- 172.21.15.144:0/2553479433 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c006030 con 0x7fc5180cd8c0 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51559a700 1 -- 172.21.15.144:0/2553479433 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0063e0 con 0x7fc5180cd8c0 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 -- 172.21.15.144:0/2553479433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 msgr2=0x7fc518140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/2553479433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc518140950 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009820 tx=0x7fc50c009b30 comp rx=0 tx=0).stop 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 -- 172.21.15.144:0/2553479433 shutdown_connections 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/2553479433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc51806fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/2553479433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc518140950 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 -- 172.21.15.144:0/2553479433 >> 172.21.15.144:0/2553479433 conn(0x7fc51806aed0 msgr2=0x7fc51806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 -- 172.21.15.144:0/2553479433 shutdown_connections 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.912+0000 7fc51cd5d700 1 -- 172.21.15.144:0/2553479433 wait complete. 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 Processor -- start 2023-11-28T11:52:38.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 -- start start 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc5181f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc5180ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5180cdc90 0x7fc51813f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5180b4f80 con 0x7fc5180cd8c0 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc518142fe0 con 0x7fc5180cdc90 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.914+0000 7fc51cd5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc518142d00 con 0x7fc51806f850 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc5180ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc5180ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36976/0 (socket says 172.21.15.144:36976) 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 -- 172.21.15.144:0/239656656 learned_addr learned my addr 172.21.15.144:0/239656656 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:38.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc51659c700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc5181f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc516d9d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5180cdc90 0x7fc51813f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 -- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 msgr2=0x7fc5181f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc5181f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 -- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5180cdc90 msgr2=0x7fc51813f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5180cdc90 0x7fc51813f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 -- 172.21.15.144:0/239656656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c009430 con 0x7fc5180cd8c0 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc515d9b700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc5180ca6d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c520 tx=0x7fc50800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508004c80 con 0x7fc5180cd8c0 2023-11-28T11:52:38.922 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc508004de0 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5080072d0 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5180b35f0 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.915+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5180b3310 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.916+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518140150 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.917+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc50801a020 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.917+0000 7fc506ffd700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc4f80682f0 0x7fc4f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.917+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fc5080857b0 con 0x7fc5180cd8c0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.917+0000 7fc51659c700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc4f80682f0 0x7fc4f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:38.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.917+0000 7fc51659c700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc4f80682f0 0x7fc4f806a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc50c013040 tx=0x7fc50c008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:38.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:38.921+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc508051fb0 con 0x7fc5180cd8c0 2023-11-28T11:52:39.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.147+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc518068690 con 0x7fc5180cd8c0 2023-11-28T11:52:39.766 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.764+0000 7fc506ffd700 1 -- 172.21.15.144:0/239656656 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc5080555d0 con 0x7fc5180cd8c0 2023-11-28T11:52:39.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc4f80682f0 msgr2=0x7fc4f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc4f80682f0 0x7fc4f806a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc50c013040 tx=0x7fc50c008070 comp rx=0 tx=0).stop 2023-11-28T11:52:39.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 msgr2=0x7fc5180ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc5180ca6d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c520 tx=0x7fc50800c830 comp rx=0 tx=0).stop 2023-11-28T11:52:39.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 shutdown_connections 2023-11-28T11:52:39.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc4f80682f0 0x7fc4f806a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5180cdc90 0x7fc51813f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc51806f850 0x7fc5181f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 --2- 172.21.15.144:0/239656656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5180cd8c0 0x7fc5180ca6d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 >> 172.21.15.144:0/239656656 conn(0x7fc51806aed0 msgr2=0x7fc5180cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 shutdown_connections 2023-11-28T11:52:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.767+0000 7fc51cd5d700 1 -- 172.21.15.144:0/239656656 wait complete. 2023-11-28T11:52:39.784 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:52:39.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.953+0000 7f53b784a700 1 Processor -- start 2023-11-28T11:52:39.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.954+0000 7f53b784a700 1 -- start start 2023-11-28T11:52:39.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.954+0000 7f53b784a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f53b00b6ae0 0x7f53b00cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b784a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b015af70 con 0x7f53b00cfa20 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b4de5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f53b00b6ae0 0x7f53b00cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b5de7700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f53b00cfa20 0x7f53b00d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53700/0 (socket says 172.21.15.144:53700) 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b784a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b00cc870 con 0x7f53b015aba0 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b5de7700 1 -- 172.21.15.144:0/2678955803 learned_addr learned my addr 172.21.15.144:0/2678955803 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b784a700 1 -- 172.21.15.144:0/2678955803 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b00cfec0 con 0x7f53b00b6ae0 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b4de5700 1 -- 172.21.15.144:0/2678955803 >> v1:172.21.15.177:6789/0 conn(0x7f53b015aba0 legacy=0x7f53b00b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b4de5700 1 -- 172.21.15.144:0/2678955803 >> v1:172.21.15.144:6789/0 conn(0x7f53b00cfa20 legacy=0x7f53b00d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b4de5700 1 -- 172.21.15.144:0/2678955803 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00ccaa0 con 0x7f53b00b6ae0 2023-11-28T11:52:39.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53b4de5700 1 --2- 172.21.15.144:0/2678955803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00b6ae0 0x7f53b00cf4e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009840 tx=0x7f53ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68c5e1118b9b2f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:39.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53a7fff700 1 -- 172.21.15.144:0/2678955803 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac011830 con 0x7f53b00b6ae0 2023-11-28T11:52:39.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53a7fff700 1 -- 172.21.15.144:0/2678955803 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac011990 con 0x7f53b00b6ae0 2023-11-28T11:52:39.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.955+0000 7f53a7fff700 1 -- 172.21.15.144:0/2678955803 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac011d20 con 0x7f53b00b6ae0 2023-11-28T11:52:39.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 -- 172.21.15.144:0/2678955803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00b6ae0 msgr2=0x7f53b00cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 --2- 172.21.15.144:0/2678955803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00b6ae0 0x7f53b00cf4e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009840 tx=0x7f53ac009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 -- 172.21.15.144:0/2678955803 shutdown_connections 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 --2- 172.21.15.144:0/2678955803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00b6ae0 0x7f53b00cf4e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 -- 172.21.15.144:0/2678955803 >> 172.21.15.144:0/2678955803 conn(0x7f53b006df70 msgr2=0x7f53b006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 -- 172.21.15.144:0/2678955803 shutdown_connections 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.956+0000 7f53b784a700 1 -- 172.21.15.144:0/2678955803 wait complete. 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.957+0000 7f53b784a700 1 Processor -- start 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.957+0000 7f53b784a700 1 -- start start 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b784a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 0x7f53b0152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b784a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53b015aba0 0x7f53b0159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b784a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53b0158320 0x7f53b015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:39.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b784a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01593f0 con 0x7f53b00cfa20 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b784a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0158870 con 0x7f53b0158320 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b784a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0158b50 con 0x7f53b015aba0 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 0x7f53b0152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 0x7f53b0152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36990/0 (socket says 172.21.15.144:36990) 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 -- 172.21.15.144:0/2348476202 learned_addr learned my addr 172.21.15.144:0/2348476202 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b5de7700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53b0158320 0x7f53b015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b4de5700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53b015aba0 0x7f53b0159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 -- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53b015aba0 msgr2=0x7f53b0159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53b015aba0 0x7f53b0159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 -- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53b0158320 msgr2=0x7f53b015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53b0158320 0x7f53b015d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:39.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 -- 172.21.15.144:0/2348476202 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0094a0 con 0x7f53b00cfa20 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b4de5700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53b015aba0 0x7f53b0159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.958+0000 7f53b55e6700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 0x7f53b0152540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c4d0 tx=0x7f53a000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.959+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a00052d0 con 0x7f53b00cfa20 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.959+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0005430 con 0x7f53b00cfa20 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.959+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b00cd060 con 0x7f53b00cfa20 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.959+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0003880 con 0x7f53b00cfa20 2023-11-28T11:52:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.959+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b00cce00 con 0x7f53b00cfa20 2023-11-28T11:52:39.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.960+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b00cd340 con 0x7f53b00cfa20 2023-11-28T11:52:39.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.960+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f53a0019030 con 0x7f53b00cfa20 2023-11-28T11:52:39.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.960+0000 7f53a5ffb700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53940682f0 0x7f539406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:39.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.960+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f53a0084ea0 con 0x7f53b00cfa20 2023-11-28T11:52:39.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.960+0000 7f53b4de5700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53940682f0 0x7f539406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:39.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.961+0000 7f53b4de5700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53940682f0 0x7f539406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0090b0 tx=0x7f53ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:39.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:39.964+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f53a004b5e0 con 0x7f53b00cfa20 2023-11-28T11:52:40.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.147+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53b0068690 con 0x7f53b00cfa20 2023-11-28T11:52:40.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.774+0000 7f53a5ffb700 1 -- 172.21.15.144:0/2348476202 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f53a00170d0 con 0x7f53b00cfa20 2023-11-28T11:52:40.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.776+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53940682f0 msgr2=0x7f539406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.776+0000 7f53b784a700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53940682f0 0x7f539406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0090b0 tx=0x7f53ac008030 comp rx=0 tx=0).stop 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.776+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 msgr2=0x7f53b0152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.776+0000 7f53b784a700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 0x7f53b0152540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c4d0 tx=0x7f53a000c7e0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 shutdown_connections 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53b00cfa20 0x7f53b0152540 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53940682f0 0x7f539406a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53b0158320 0x7f53b015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 --2- 172.21.15.144:0/2348476202 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53b015aba0 0x7f53b0159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 >> 172.21.15.144:0/2348476202 conn(0x7f53b006df70 msgr2=0x7f53b00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:40.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 shutdown_connections 2023-11-28T11:52:40.779 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.777+0000 7f53b784a700 1 -- 172.21.15.144:0/2348476202 wait complete. 2023-11-28T11:52:40.779 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:52:40.794 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:52:40.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.923+0000 7fc276ef1700 1 Processor -- start 2023-11-28T11:52:40.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.923+0000 7fc276ef1700 1 -- start start 2023-11-28T11:52:40.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc276ef1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2700af430 0x7fc270070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc276ef1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2700713c0 con 0x7fc2700af060 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc276ef1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2700716a0 con 0x7fc27006a980 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc276ef1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc270071980 con 0x7fc2700af430 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc27548e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2700af430 0x7fc270070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc274c8d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc2700af060 0x7fc27006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53710/0 (socket says 172.21.15.144:53710) 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc274c8d700 1 -- 172.21.15.144:0/546433288 learned_addr learned my addr 172.21.15.144:0/546433288 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc27548e700 1 -- 172.21.15.144:0/546433288 >> v1:172.21.15.177:6789/0 conn(0x7fc27006a980 legacy=0x7fc2700ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc27548e700 1 -- 172.21.15.144:0/546433288 >> v1:172.21.15.144:6789/0 conn(0x7fc2700af060 legacy=0x7fc27006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc27548e700 1 -- 172.21.15.144:0/546433288 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270071c60 con 0x7fc2700af430 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.924+0000 7fc26f7fe700 1 -- 172.21.15.144:0/546433288 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619979980 0 0) 0x7fc2700713c0 con 0x7fc2700af060 2023-11-28T11:52:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc27548e700 1 --2- 172.21.15.144:0/546433288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270070d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37688ff28616fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:40.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc26f7fe700 1 -- 172.21.15.144:0/546433288 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800e070 con 0x7fc2700af430 2023-11-28T11:52:40.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc26f7fe700 1 -- 172.21.15.144:0/546433288 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26800bbd0 con 0x7fc2700af430 2023-11-28T11:52:40.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc26f7fe700 1 -- 172.21.15.144:0/546433288 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268012430 con 0x7fc2700af430 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 -- 172.21.15.144:0/546433288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 msgr2=0x7fc270070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 --2- 172.21.15.144:0/546433288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270070d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26800b330 comp rx=0 tx=0).stop 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 -- 172.21.15.144:0/546433288 shutdown_connections 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 --2- 172.21.15.144:0/546433288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270070d50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 -- 172.21.15.144:0/546433288 >> 172.21.15.144:0/546433288 conn(0x7fc27006b380 msgr2=0x7fc27006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 -- 172.21.15.144:0/546433288 shutdown_connections 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.925+0000 7fc276ef1700 1 -- 172.21.15.144:0/546433288 wait complete. 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.927+0000 7fc276ef1700 1 Processor -- start 2023-11-28T11:52:40.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.927+0000 7fc276ef1700 1 -- start start 2023-11-28T11:52:40.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc276ef1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc27006a980 0x7fc2700b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:40.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc276ef1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2700af060 0x7fc2700b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:40.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc276ef1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:40.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc276ef1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27015e420 con 0x7fc2700af430 2023-11-28T11:52:40.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc276ef1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27015fec0 con 0x7fc27006a980 2023-11-28T11:52:40.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc276ef1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27015db30 con 0x7fc2700af060 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37008/0 (socket says 172.21.15.144:37008) 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 -- 172.21.15.144:0/1749044252 learned_addr learned my addr 172.21.15.144:0/1749044252 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc274c8d700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc27006a980 0x7fc2700b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc26ffff700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2700af060 0x7fc2700b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 -- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2700af060 msgr2=0x7fc2700b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2700af060 0x7fc2700b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 -- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc27006a980 msgr2=0x7fc2700b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc27006a980 0x7fc2700b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 -- 172.21.15.144:0/1749044252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc268009a70 con 0x7fc2700af430 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc26ffff700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2700af060 0x7fc2700b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc27548e700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270155bc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc268014040 tx=0x7fc268004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.928+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268012430 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.929+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268020070 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.929+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27006d9a0 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.929+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268007000 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.929+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc270071f40 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.929+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27006db30 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.930+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc268004280 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.930+0000 7fc26d7fa700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2540682f0 0x7fc25406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.930+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fc268022070 con 0x7fc2700af430 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.930+0000 7fc274c8d700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2540682f0 0x7fc25406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:40.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.931+0000 7fc274c8d700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2540682f0 0x7fc25406a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc264007b10 tx=0x7fc264007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:40.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:40.935+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2680549e0 con 0x7fc2700af430 2023-11-28T11:52:41.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.118+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc270068690 con 0x7fc2700af430 2023-11-28T11:52:41.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.784+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1749044252 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc268058000 con 0x7fc2700af430 2023-11-28T11:52:41.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2540682f0 msgr2=0x7fc25406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2540682f0 0x7fc25406a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc264007b10 tx=0x7fc264007a60 comp rx=0 tx=0).stop 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 msgr2=0x7fc270155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270155bc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc268014040 tx=0x7fc268004ad0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 shutdown_connections 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700af430 0x7fc270155bc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2540682f0 0x7fc25406a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc27006a980 0x7fc2700b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.787+0000 7fc276ef1700 1 --2- 172.21.15.144:0/1749044252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2700af060 0x7fc2700b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.788+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 >> 172.21.15.144:0/1749044252 conn(0x7fc27006b380 msgr2=0x7fc27015c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:41.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.788+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 shutdown_connections 2023-11-28T11:52:41.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.788+0000 7fc276ef1700 1 -- 172.21.15.144:0/1749044252 wait complete. 2023-11-28T11:52:41.790 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' removed 2023-11-28T11:52:41.802 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:52:41.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.926+0000 7fc30c084700 1 Processor -- start 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.926+0000 7fc30c084700 1 -- start start 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30c084700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc30415cef0 0x7fc3040af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30c084700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30c084700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc30406b8c0 0x7fc304071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30c084700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30406bd40 con 0x7fc30406b8c0 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30c084700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040cc5f0 con 0x7fc30415cef0 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30c084700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415d470 con 0x7fc3040cbdf0 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30a621700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc30406b8c0 0x7fc304071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30a621700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc30406b8c0 0x7fc304071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37016/0 (socket says 172.21.15.144:37016) 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.927+0000 7fc30a621700 1 -- 172.21.15.144:0/3118627704 learned_addr learned my addr 172.21.15.144:0/3118627704 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30961f700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc309e20700 1 --2- 172.21.15.144:0/3118627704 >> v2:172.21.15.177:3300/0 conn(0x7fc30415cef0 0x7fc3040af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30a621700 1 -- 172.21.15.144:0/3118627704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 msgr2=0x7fc30406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30a621700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30406b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30a621700 1 -- 172.21.15.144:0/3118627704 >> v2:172.21.15.177:3300/0 conn(0x7fc30415cef0 msgr2=0x7fc3040af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30a621700 1 --2- 172.21.15.144:0/3118627704 >> v2:172.21.15.177:3300/0 conn(0x7fc30415cef0 0x7fc3040af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30a621700 1 -- 172.21.15.144:0/3118627704 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30415d750 con 0x7fc30406b8c0 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30961f700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc309e20700 1 --2- 172.21.15.144:0/3118627704 >> v2:172.21.15.177:3300/0 conn(0x7fc30415cef0 0x7fc3040af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc30a621700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30406b8c0 0x7fc304071040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc3040723f0 tx=0x7fc2fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dd8b9e848047b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc308e1e700 1 -- 172.21.15.144:0/3118627704 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00e070 con 0x7fc30406b8c0 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc308e1e700 1 -- 172.21.15.144:0/3118627704 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00b9c0 con 0x7fc30406b8c0 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.928+0000 7fc308e1e700 1 -- 172.21.15.144:0/3118627704 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00bd70 con 0x7fc30406b8c0 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 -- 172.21.15.144:0/3118627704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30406b8c0 msgr2=0x7fc304071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:41.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30406b8c0 0x7fc304071040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc3040723f0 tx=0x7fc2fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 -- 172.21.15.144:0/3118627704 shutdown_connections 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30406b8c0 0x7fc304071040 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 --2- 172.21.15.144:0/3118627704 >> v2:172.21.15.177:3300/0 conn(0x7fc30415cef0 0x7fc3040af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 --2- 172.21.15.144:0/3118627704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 -- 172.21.15.144:0/3118627704 >> 172.21.15.144:0/3118627704 conn(0x7fc3040d0000 msgr2=0x7fc3040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 -- 172.21.15.144:0/3118627704 shutdown_connections 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.929+0000 7fc30c084700 1 -- 172.21.15.144:0/3118627704 wait complete. 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.930+0000 7fc30c084700 1 Processor -- start 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.930+0000 7fc30c084700 1 -- start start 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc30406b8c0 0x7fc30414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 0x7fc3040b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040c9190 con 0x7fc30415cef0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040b4070 con 0x7fc30406b8c0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30414f9d0 con 0x7fc3040cbdf0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 0x7fc3040b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 0x7fc3040b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37022/0 (socket says 172.21.15.144:37022) 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 -- 172.21.15.144:0/3861260341 learned_addr learned my addr 172.21.15.144:0/3861260341 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc309e20700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc30406b8c0 0x7fc30414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30961f700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 -- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 msgr2=0x7fc30414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 -- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc30406b8c0 msgr2=0x7fc30414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc30406b8c0 0x7fc30414f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 -- 172.21.15.144:0/3861260341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc009b30 con 0x7fc30415cef0 2023-11-28T11:52:41.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc309e20700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc30406b8c0 0x7fc30414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30a621700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 0x7fc3040b5440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc30406c7b0 tx=0x7fc2fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc004960 con 0x7fc30415cef0 2023-11-28T11:52:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc0099b0 con 0x7fc30415cef0 2023-11-28T11:52:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc01b650 con 0x7fc30415cef0 2023-11-28T11:52:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30415dd10 con 0x7fc30415cef0 2023-11-28T11:52:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.931+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30415da30 con 0x7fc30415cef0 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.932+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30415dff0 con 0x7fc30415cef0 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.933+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2fc02b070 con 0x7fc30415cef0 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.933+0000 7fc2fa7fc700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2e80682f0 0x7fc2e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.933+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc2fc086e80 con 0x7fc30415cef0 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.933+0000 7fc309e20700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2e80682f0 0x7fc2e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.933+0000 7fc309e20700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2e80682f0 0x7fc2e806a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4006fd0 tx=0x7fc2f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:41.937+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2fc0539e0 con 0x7fc30415cef0 2023-11-28T11:52:42.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.123+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc30409dcf0 con 0x7fc30415cef0 2023-11-28T11:52:42.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.128+0000 7fc2fa7fc700 1 -- 172.21.15.144:0/3861260341 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc2fc057000 con 0x7fc30415cef0 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2e80682f0 msgr2=0x7fc2e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2e80682f0 0x7fc2e806a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4006fd0 tx=0x7fc2f4008040 comp rx=0 tx=0).stop 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 msgr2=0x7fc3040b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 0x7fc3040b5440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc30406c7b0 tx=0x7fc2fc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 shutdown_connections 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30415cef0 0x7fc3040b5440 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2e80682f0 0x7fc2e806a7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc30406b8c0 0x7fc30414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 --2- 172.21.15.144:0/3861260341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3040cbdf0 0x7fc30414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 >> 172.21.15.144:0/3861260341 conn(0x7fc3040d0000 msgr2=0x7fc30406a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 shutdown_connections 2023-11-28T11:52:42.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.131+0000 7fc30c084700 1 -- 172.21.15.144:0/3861260341 wait complete. 2023-11-28T11:52:42.143 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:52:42.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.263+0000 7f16d0d82700 1 Processor -- start 2023-11-28T11:52:42.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.263+0000 7f16d0d82700 1 -- start start 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16d0d82700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16cc0d0a00 0x7f16cc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16d0d82700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0b2930 0x7f16cc0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16d0d82700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc06b420 con 0x7f16cc0ce040 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16d0d82700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc0b32f0 con 0x7f16cc0d0a00 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16d0d82700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc06ff60 con 0x7f16cc0b2930 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c9d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16cc0d0a00 0x7f16cc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16ca59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f16cc0ce040 0x7f16cc06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53738/0 (socket says 172.21.15.144:53738) 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16ca59c700 1 -- 172.21.15.144:0/623483697 learned_addr learned my addr 172.21.15.144:0/623483697 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:42.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16cad9d700 1 --2- 172.21.15.144:0/623483697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0b2930 0x7f16cc0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c9d9b700 1 -- 172.21.15.144:0/623483697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0b2930 msgr2=0x7f16cc0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c9d9b700 1 --2- 172.21.15.144:0/623483697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0b2930 0x7f16cc0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c9d9b700 1 -- 172.21.15.144:0/623483697 >> v1:172.21.15.144:6789/0 conn(0x7f16cc0ce040 legacy=0x7f16cc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c959a700 1 -- 172.21.15.144:0/623483697 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225601514 0 0) 0x7f16cc06b420 con 0x7f16cc0ce040 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c9d9b700 1 -- 172.21.15.144:0/623483697 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc06b700 con 0x7f16cc0d0a00 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c9d9b700 1 --2- 172.21.15.144:0/623483697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0d0a00 0x7f16cc15df10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009840 tx=0x7f16c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca28d17c530261b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.264+0000 7f16c959a700 1 -- 172.21.15.144:0/623483697 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0011830 con 0x7f16cc0d0a00 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16c959a700 1 -- 172.21.15.144:0/623483697 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c0011990 con 0x7f16cc0d0a00 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16c959a700 1 -- 172.21.15.144:0/623483697 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0011d20 con 0x7f16cc0d0a00 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 -- 172.21.15.144:0/623483697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0d0a00 msgr2=0x7f16cc15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 --2- 172.21.15.144:0/623483697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0d0a00 0x7f16cc15df10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009840 tx=0x7f16c0009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 -- 172.21.15.144:0/623483697 shutdown_connections 2023-11-28T11:52:42.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 --2- 172.21.15.144:0/623483697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0b2930 0x7f16cc0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 --2- 172.21.15.144:0/623483697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0d0a00 0x7f16cc15df10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 -- 172.21.15.144:0/623483697 >> 172.21.15.144:0/623483697 conn(0x7f16cc06aed0 msgr2=0x7f16cc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:42.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 -- 172.21.15.144:0/623483697 shutdown_connections 2023-11-28T11:52:42.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.265+0000 7f16d0d82700 1 -- 172.21.15.144:0/623483697 wait complete. 2023-11-28T11:52:42.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.267+0000 7f16d0d82700 1 Processor -- start 2023-11-28T11:52:42.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.267+0000 7f16d0d82700 1 -- start start 2023-11-28T11:52:42.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16d0d82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 0x7f16cc0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16d0d82700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc0ce040 0x7f16cc1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16d0d82700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0d0a00 0x7f16cc0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16d0d82700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c8770 con 0x7f16cc0b2930 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16d0d82700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c6250 con 0x7f16cc0ce040 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16d0d82700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c6530 con 0x7f16cc0d0a00 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 0x7f16cc0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 0x7f16cc0c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37032/0 (socket says 172.21.15.144:37032) 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 -- 172.21.15.144:0/475460479 learned_addr learned my addr 172.21.15.144:0/475460479 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 -- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0d0a00 msgr2=0x7f16cc0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16cad9d700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0d0a00 0x7f16cc0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16c9d9b700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc0ce040 0x7f16cc1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0d0a00 0x7f16cc0cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 -- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc0ce040 msgr2=0x7f16cc1f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc0ce040 0x7f16cc1f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 -- 172.21.15.144:0/475460479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00094a0 con 0x7f16cc0b2930 2023-11-28T11:52:42.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16c9d9b700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc0ce040 0x7f16cc1f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.268+0000 7f16ca59c700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 0x7f16cc0c8c00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c4a0 tx=0x7f16bc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.269+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0074e0 con 0x7f16cc0b2930 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.269+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc007640 con 0x7f16cc0b2930 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.269+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc06bcc0 con 0x7f16cc0b2930 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.269+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc005a70 con 0x7f16cc0b2930 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.269+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc06b9e0 con 0x7f16cc0b2930 2023-11-28T11:52:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.270+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc06bfa0 con 0x7f16cc0b2930 2023-11-28T11:52:42.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.270+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f16bc019030 con 0x7f16cc0b2930 2023-11-28T11:52:42.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.270+0000 7f16baffd700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16ac0682f0 0x7f16ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.270+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f16bc084db0 con 0x7f16cc0b2930 2023-11-28T11:52:42.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.270+0000 7f16c9d9b700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16ac0682f0 0x7f16ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.271+0000 7f16c9d9b700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16ac0682f0 0x7f16ac06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0caad0 tx=0x7f16c0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:42.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.274+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f16bc051890 con 0x7f16cc0b2930 2023-11-28T11:52:42.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.528+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f16cc0c7f60 con 0x7f16cc0b2930 2023-11-28T11:52:42.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.784+0000 7f16baffd700 1 -- 172.21.15.144:0/475460479 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v95) v1 ==== 142+0+0 (secure 0 0 0) 0x7f16bc0170d0 con 0x7f16cc0b2930 2023-11-28T11:52:42.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.786+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16ac0682f0 msgr2=0x7f16ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.786+0000 7f16d0d82700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16ac0682f0 0x7f16ac06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f16cc0caad0 tx=0x7f16c0008030 comp rx=0 tx=0).stop 2023-11-28T11:52:42.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.786+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 msgr2=0x7f16cc0c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.786+0000 7f16d0d82700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 0x7f16cc0c8c00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c4a0 tx=0x7f16bc00c860 comp rx=0 tx=0).stop 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.786+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 shutdown_connections 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16cc0b2930 0x7f16cc0c8c00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16ac0682f0 0x7f16ac06a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0d0a00 0x7f16cc0cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 --2- 172.21.15.144:0/475460479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc0ce040 0x7f16cc1f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 >> 172.21.15.144:0/475460479 conn(0x7f16cc06aed0 msgr2=0x7f16cc15e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 shutdown_connections 2023-11-28T11:52:42.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.787+0000 7f16d0d82700 1 -- 172.21.15.144:0/475460479 wait complete. 2023-11-28T11:52:42.800 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T11:52:42.800 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T11:52:42.801 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:52:42.801 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:52:42.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.927+0000 7fd8a7396700 1 Processor -- start 2023-11-28T11:52:42.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.927+0000 7fd8a7396700 1 -- start start 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a7396700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd8a006d4f0 0x7fd8a00cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a7396700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a00cda60 con 0x7fd8a015dae0 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a7396700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a015e050 con 0x7fd8a0159650 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a7396700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a00ce630 con 0x7fd8a006d4f0 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a5933700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd8a015dae0 0x7fd8a00c51f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53754/0 (socket says 172.21.15.144:53754) 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a5933700 1 -- 172.21.15.144:0/724607591 learned_addr learned my addr 172.21.15.144:0/724607591 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457616396 0 0) 0x7fd8a00cda60 con 0x7fd8a015dae0 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd88c0035b0 con 0x7fd8a015dae0 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd8a4931700 1 --2- 172.21.15.144:0/724607591 >> v2:172.21.15.177:3300/0 conn(0x7fd8a006d4f0 0x7fd8a00cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372706118 0 0) 0x7fd88c0035b0 con 0x7fd8a015dae0 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 >> v2:172.21.15.177:3300/0 conn(0x7fd8a006d4f0 msgr2=0x7fd8a00cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:42.932 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 --2- 172.21.15.144:0/724607591 >> v2:172.21.15.177:3300/0 conn(0x7fd8a006d4f0 0x7fd8a00cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 >> v1:172.21.15.177:6789/0 conn(0x7fd8a0159650 legacy=0x7fd8a006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.928+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00cdc90 con 0x7fd8a015dae0 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fd8980019f0 con 0x7fd8a015dae0 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 >> v1:172.21.15.144:6789/0 conn(0x7fd8a015dae0 legacy=0x7fd8a00c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd88c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd88c004850 0x7fd88c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd88c005200 0x7fd88c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00ce630 con 0x7fd88c003a00 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c00c060 con 0x7fd88c005200 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c00c220 con 0x7fd88c004850 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd88c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5933700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd88c005200 0x7fd88c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.933 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a4931700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd88c004850 0x7fd88c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 -- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd88c004850 msgr2=0x7fd88c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd88c004850 0x7fd88c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 -- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd88c005200 msgr2=0x7fd88c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd88c005200 0x7fd88c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 -- 172.21.15.144:0/724607591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0035b0 con 0x7fd88c003a00 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5933700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd88c005200 0x7fd88c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5933700 1 -- 172.21.15.144:0/724607591 reap_dead start 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd8a5132700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd88c003e10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0017c0 tx=0x7fd88c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f0a84b33ea175 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0052d0 con 0x7fd88c003a00 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c005430 con 0x7fd88c003a00 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.929+0000 7fd897fff700 1 -- 172.21.15.144:0/724607591 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0038c0 con 0x7fd88c003a00 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 -- 172.21.15.144:0/724607591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 msgr2=0x7fd88c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd88c003e10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0017c0 tx=0x7fd88c005880 comp rx=0 tx=0).stop 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 -- 172.21.15.144:0/724607591 shutdown_connections 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 --2- 172.21.15.144:0/724607591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd88c003e10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 -- 172.21.15.144:0/724607591 >> 172.21.15.144:0/724607591 conn(0x7fd8a00b19f0 msgr2=0x7fd8a00b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 -- 172.21.15.144:0/724607591 shutdown_connections 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.930+0000 7fd8a7396700 1 -- 172.21.15.144:0/724607591 wait complete. 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.931+0000 7fd8a7396700 1 Processor -- start 2023-11-28T11:52:42.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.931+0000 7fd8a7396700 1 -- start start 2023-11-28T11:52:42.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a7396700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd8a014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a7396700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8a00b56b0 0x7fd8a00c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a7396700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8a00c60a0 0x7fd8a01500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a7396700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a00c6540 con 0x7fd88c003a00 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a7396700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a0155990 con 0x7fd8a00b56b0 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a7396700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a014ed00 con 0x7fd8a00c60a0 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd8a014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd8a014e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37052/0 (socket says 172.21.15.144:37052) 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 -- 172.21.15.144:0/4237596424 learned_addr learned my addr 172.21.15.144:0/4237596424 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a4931700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8a00b56b0 0x7fd8a00c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5933700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8a00c60a0 0x7fd8a01500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 -- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8a00c60a0 msgr2=0x7fd8a01500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8a00c60a0 0x7fd8a01500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 -- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8a00b56b0 msgr2=0x7fd8a00c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8a00b56b0 0x7fd8a00c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 -- 172.21.15.144:0/4237596424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00ce250 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5933700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8a00c60a0 0x7fd8a01500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd8a5132700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd8a014e410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd89c015040 tx=0x7fd89c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c001290 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.932+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c01d480 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.933+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c01d810 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.933+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a00cdf70 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.933+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a00c4e30 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.934+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a0068690 con 0x7fd88c003a00 2023-11-28T11:52:42.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.934+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd89c02f070 con 0x7fd88c003a00 2023-11-28T11:52:42.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.934+0000 7fd895ffb700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8840682f0 0x7fd88406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:42.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.934+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd89c0866b0 con 0x7fd88c003a00 2023-11-28T11:52:42.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.934+0000 7fd8a4931700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8840682f0 0x7fd88406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:42.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.935+0000 7fd8a4931700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8840682f0 0x7fd88406a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0151c80 tx=0x7fd890008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:42.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:42.938+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd89c053210 con 0x7fd88c003a00 2023-11-28T11:52:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.196+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fd8a015c950 con 0x7fd88c003a00 2023-11-28T11:52:43.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.783+0000 7fd895ffb700 1 -- 172.21.15.144:0/4237596424 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v96) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd89c056830 con 0x7fd88c003a00 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8840682f0 msgr2=0x7fd88406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8840682f0 0x7fd88406a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0151c80 tx=0x7fd890008040 comp rx=0 tx=0).stop 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 msgr2=0x7fd8a014e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd8a014e410 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd89c015040 tx=0x7fd89c001120 comp rx=0 tx=0).stop 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 shutdown_connections 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88c003a00 0x7fd8a014e410 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd8840682f0 0x7fd88406a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8a00c60a0 0x7fd8a01500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 --2- 172.21.15.144:0/4237596424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8a00b56b0 0x7fd8a00c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 >> 172.21.15.144:0/4237596424 conn(0x7fd8a00b19f0 msgr2=0x7fd8a0073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:43.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 shutdown_connections 2023-11-28T11:52:43.790 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.786+0000 7fd8a7396700 1 -- 172.21.15.144:0/4237596424 wait complete. 2023-11-28T11:52:43.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:52:43.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.927+0000 7f824b504700 1 Processor -- start 2023-11-28T11:52:43.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f824b504700 1 -- start start 2023-11-28T11:52:43.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f824b504700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f824415cef0 0x7f82440af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:43.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f824b504700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82440cbdf0 0x7f824406b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:43.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f824b504700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440cc5f0 con 0x7f824406b8c0 2023-11-28T11:52:43.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f824b504700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824406bd40 con 0x7f824415cef0 2023-11-28T11:52:43.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f824b504700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824415d470 con 0x7f82440cbdf0 2023-11-28T11:52:43.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.928+0000 7f82492a0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f824415cef0 0x7f82440af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f824415cef0 0x7f82440af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37060/0 (socket says 172.21.15.144:37060) 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 -- 172.21.15.144:0/3001695572 learned_addr learned my addr 172.21.15.144:0/3001695572 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f8248a9f700 1 --2- 172.21.15.144:0/3001695572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82440cbdf0 0x7f824406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 -- 172.21.15.144:0/3001695572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82440cbdf0 msgr2=0x7f824406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 --2- 172.21.15.144:0/3001695572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82440cbdf0 0x7f824406b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 -- 172.21.15.144:0/3001695572 >> v1:172.21.15.177:6789/0 conn(0x7f824406b8c0 legacy=0x7f8244071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 -- 172.21.15.144:0/3001695572 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824415d750 con 0x7f824415cef0 2023-11-28T11:52:43.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f82492a0700 1 --2- 172.21.15.144:0/3001695572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824415cef0 0x7f82440af400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f823400bb90 tx=0x7f823400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cfaa23a01abf70 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f823bfff700 1 -- 172.21.15.144:0/3001695572 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400dd10 con 0x7f824415cef0 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f823bfff700 1 -- 172.21.15.144:0/3001695572 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400de70 con 0x7f824415cef0 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.929+0000 7f823bfff700 1 -- 172.21.15.144:0/3001695572 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234004230 con 0x7f824415cef0 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 -- 172.21.15.144:0/3001695572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824415cef0 msgr2=0x7f82440af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 --2- 172.21.15.144:0/3001695572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824415cef0 0x7f82440af400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f823400bb90 tx=0x7f823400bea0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 -- 172.21.15.144:0/3001695572 shutdown_connections 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 --2- 172.21.15.144:0/3001695572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824415cef0 0x7f82440af400 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 --2- 172.21.15.144:0/3001695572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82440cbdf0 0x7f824406b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 -- 172.21.15.144:0/3001695572 >> 172.21.15.144:0/3001695572 conn(0x7f82440d0000 msgr2=0x7f82440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 -- 172.21.15.144:0/3001695572 shutdown_connections 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.930+0000 7f824b504700 1 -- 172.21.15.144:0/3001695572 wait complete. 2023-11-28T11:52:43.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.931+0000 7f824b504700 1 Processor -- start 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.931+0000 7f824b504700 1 -- start start 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f824406b8c0 0x7f824409d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 0x7f82440c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f824415cef0 0x7f82440c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8244150c90 con 0x7f82440cbdf0 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82440c70d0 con 0x7f824406b8c0 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8244150330 con 0x7f824415cef0 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 0x7f82440c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 0x7f82440c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37068/0 (socket says 172.21.15.144:37068) 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 -- 172.21.15.144:0/1171771439 learned_addr learned my addr 172.21.15.144:0/1171771439 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8249aa1700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f824415cef0 0x7f82440c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:43.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f82492a0700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f824406b8c0 0x7f824409d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 -- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f824415cef0 msgr2=0x7f82440c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f824415cef0 0x7f82440c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 -- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f824406b8c0 msgr2=0x7f824409d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f824406b8c0 0x7f824409d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 -- 172.21.15.144:0/1171771439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823400b7f0 con 0x7f82440cbdf0 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f82492a0700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f824406b8c0 0x7f824409d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8248a9f700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 0x7f82440c9660 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f824000c820 tx=0x7f824000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240011800 con 0x7f82440cbdf0 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8240011960 con 0x7f82440cbdf0 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824415dd10 con 0x7f82440cbdf0 2023-11-28T11:52:43.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.932+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240011d10 con 0x7f82440cbdf0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.933+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824415da30 con 0x7f82440cbdf0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.934+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824415dff0 con 0x7f82440cbdf0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.934+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8240021070 con 0x7f82440cbdf0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.934+0000 7f8239ffb700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82280682f0 0x7f822806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.934+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8240085830 con 0x7f82440cbdf0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.934+0000 7f82492a0700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82280682f0 0x7f822806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.935+0000 7f82492a0700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82280682f0 0x7f822806a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f823400b340 tx=0x7f8234013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:43.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:43.938+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8240052390 con 0x7f82440cbdf0 2023-11-28T11:52:44.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.121+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f82440d0e60 con 0x7f82440cbdf0 2023-11-28T11:52:44.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.125+0000 7f8239ffb700 1 -- 172.21.15.144:0/1171771439 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82400150c0 con 0x7f82440cbdf0 2023-11-28T11:52:44.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82280682f0 msgr2=0x7f822806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82280682f0 0x7f822806a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f823400b340 tx=0x7f8234013000 comp rx=0 tx=0).stop 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 msgr2=0x7f82440c9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 0x7f82440c9660 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f824000c820 tx=0x7f824000cb30 comp rx=0 tx=0).stop 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 shutdown_connections 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f82280682f0 0x7f822806a7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f824415cef0 0x7f82440c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f824406b8c0 0x7f824409d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 --2- 172.21.15.144:0/1171771439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82440cbdf0 0x7f82440c9660 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.127+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 >> 172.21.15.144:0/1171771439 conn(0x7f82440d0000 msgr2=0x7f82440aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.128+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 shutdown_connections 2023-11-28T11:52:44.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.128+0000 7f824b504700 1 -- 172.21.15.144:0/1171771439 wait complete. 2023-11-28T11:52:44.140 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:52:44.140 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:52:44.193 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:52:44.238 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:52:44.238 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:44.238 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:44.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:52:44.256 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:52:44.256 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:44.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:52:44.298 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:52:44.298 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:44.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:52:44.315 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:52:44.315 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:52:44.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:52:44.357 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:52:44.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:44.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.476+0000 7fad898d1700 1 Processor -- start 2023-11-28T11:52:44.477 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.476+0000 7fad898d1700 1 -- start start 2023-11-28T11:52:44.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad898d1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fad84161340 0x7fad840d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad898d1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 0x7fad840cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad898d1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840d1950 con 0x7fad8415c9b0 2023-11-28T11:52:44.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad898d1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad840b5800 con 0x7fad84161340 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad898d1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840b5300 con 0x7fad840d2320 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad82ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fad8415c9b0 0x7fad840b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53786/0 (socket says 172.21.15.144:53786) 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad82ffd700 1 -- 172.21.15.144:0/2551175079 learned_addr learned my addr 172.21.15.144:0/2551175079 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad837fe700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 0x7fad840cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad827fc700 1 --2- 172.21.15.144:0/2551175079 >> v2:172.21.15.177:3300/0 conn(0x7fad84161340 0x7fad840d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189776992 0 0) 0x7fad840d1950 con 0x7fad8415c9b0 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.477+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c0035b0 con 0x7fad8415c9b0 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2008100972 0 0) 0x7fad6c0035b0 con 0x7fad8415c9b0 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 msgr2=0x7fad840cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 0x7fad840cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.482 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 >> v2:172.21.15.177:3300/0 conn(0x7fad84161340 msgr2=0x7fad840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 --2- 172.21.15.144:0/2551175079 >> v2:172.21.15.177:3300/0 conn(0x7fad84161340 0x7fad840d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840b5520 con 0x7fad8415c9b0 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fad78001670 con 0x7fad8415c9b0 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 >> v1:172.21.15.144:6789/0 conn(0x7fad8415c9b0 legacy=0x7fad840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad6c003a00 0x7fad6c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad6c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 0x7fad6c005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840b5800 con 0x7fad6c004430 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840b5300 con 0x7fad6c003a00 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00ba10 con 0x7fad840d2320 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad6c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad82ffd700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad6c003a00 0x7fad6c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad837fe700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 0x7fad6c005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 -- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 msgr2=0x7fad6c005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:44.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840d2320 0x7fad6c005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 -- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad6c003a00 msgr2=0x7fad6c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:44.486 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad6c003a00 0x7fad6c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 -- 172.21.15.144:0/2551175079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0035b0 con 0x7fad6c004430 2023-11-28T11:52:44.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad827fc700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad6c0048f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad74009b90 tx=0x7fad6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd173aaa4bbfa80f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:44.487 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400e6c0 con 0x7fad6c004430 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7400e820 con 0x7fad6c004430 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.478+0000 7fad81ffb700 1 -- 172.21.15.144:0/2551175079 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400eb90 con 0x7fad6c004430 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad898d1700 1 -- 172.21.15.144:0/2551175079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 msgr2=0x7fad6c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad898d1700 1 --2- 172.21.15.144:0/2551175079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad6c0048f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad74009b90 tx=0x7fad6c005880 comp rx=0 tx=0).stop 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad837fe700 1 -- 172.21.15.144:0/2551175079 reap_dead start 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad898d1700 1 -- 172.21.15.144:0/2551175079 shutdown_connections 2023-11-28T11:52:44.488 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad898d1700 1 -- 172.21.15.144:0/2551175079 >> 172.21.15.144:0/2551175079 conn(0x7fad84072600 msgr2=0x7fad840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:44.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad898d1700 1 -- 172.21.15.144:0/2551175079 shutdown_connections 2023-11-28T11:52:44.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.479+0000 7fad898d1700 1 -- 172.21.15.144:0/2551175079 wait complete. 2023-11-28T11:52:44.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.481+0000 7fad898d1700 1 Processor -- start 2023-11-28T11:52:44.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.481+0000 7fad898d1700 1 -- start start 2023-11-28T11:52:44.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad84157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840c76d0 0x7fad841e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad84156a90 0x7fad84160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad841e8880 con 0x7fad6c004430 2023-11-28T11:52:44.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840c7b40 con 0x7fad84156a90 2023-11-28T11:52:44.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840c6ca0 con 0x7fad840c76d0 2023-11-28T11:52:44.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad84157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad84157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37086/0 (socket says 172.21.15.144:37086) 2023-11-28T11:52:44.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 -- 172.21.15.144:0/2645862641 learned_addr learned my addr 172.21.15.144:0/2645862641 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:44.495 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad837fe700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad84156a90 0x7fad84160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.496 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad827fc700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840c76d0 0x7fad841e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 -- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840c76d0 msgr2=0x7fad841e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840c76d0 0x7fad841e7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 -- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad84156a90 msgr2=0x7fad84160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad84156a90 0x7fad84160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 -- 172.21.15.144:0/2645862641 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840d15e0 con 0x7fad6c004430 2023-11-28T11:52:44.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad837fe700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad84156a90 0x7fad84160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad827fc700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840c76d0 0x7fad841e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad82ffd700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad84157250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad78001aa0 tx=0x7fad780075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad780077d0 con 0x7fad6c004430 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7800f8a0 con 0x7fad6c004430 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800fc50 con 0x7fad6c004430 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad840d1300 con 0x7fad6c004430 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.482+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8415cd80 con 0x7fad6c004430 2023-11-28T11:52:44.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.483+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840745c0 con 0x7fad6c004430 2023-11-28T11:52:44.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.484+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad7800fdb0 con 0x7fad6c004430 2023-11-28T11:52:44.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.484+0000 7fad6bfff700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad700682f0 0x7fad7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.484+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fad78085b00 con 0x7fad6c004430 2023-11-28T11:52:44.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.484+0000 7fad827fc700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad700682f0 0x7fad7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.484+0000 7fad827fc700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad700682f0 0x7fad7006a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fad840c8ac0 tx=0x7fad74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:44.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.488+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad78052660 con 0x7fad6c004430 2023-11-28T11:52:44.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.714+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad840d08a0 con 0x7fad6c004430 2023-11-28T11:52:44.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.714+0000 7fad6bfff700 1 -- 172.21.15.144:0/2645862641 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fad78055c80 con 0x7fad6c004430 2023-11-28T11:52:44.715 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:44.715 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3111014256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3111014256},{"type":"v1","addr":"172.21.15.177:6835","nonce":3111014256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/609277252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":609277252},{"type":"v1","addr":"172.21.15.144:6837","nonce":609277252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/1190741768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1190741768},{"type":"v1","addr":"172.21.15.177:6834","nonce":1190741768}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5454,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1891472026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1891472026},{"type":"v1","addr":"172.21.15.144:6836","nonce":1891472026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad700682f0 msgr2=0x7fad7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad700682f0 0x7fad7006a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fad840c8ac0 tx=0x7fad74008030 comp rx=0 tx=0).stop 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 msgr2=0x7fad84157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad84157250 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad78001aa0 tx=0x7fad780075f0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 shutdown_connections 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c004430 0x7fad84157250 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad700682f0 0x7fad7006a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad84156a90 0x7fad84160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 --2- 172.21.15.144:0/2645862641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad840c76d0 0x7fad841e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 >> 172.21.15.144:0/2645862641 conn(0x7fad84072600 msgr2=0x7fad8415f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.716+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 shutdown_connections 2023-11-28T11:52:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.717+0000 7fad898d1700 1 -- 172.21.15.144:0/2645862641 wait complete. 2023-11-28T11:52:44.719 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 18 2023-11-28T11:52:44.730 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:52:44.730 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:52:44.730 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:52:44.730 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:44.731 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:52:44.731 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:52:44.731 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:52:44.731 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:44.731 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:52:44.731 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:52:44.731 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:52:44.732 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:44.732 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:52:44.732 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:52:44.732 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:52:44.732 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:52:44.739 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:52:44.740 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:52:44.741 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:52:44.741 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:52:44.743 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:52:44.743 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:52:44.744 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:52:44.744 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:52:44.782 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:52:44.782 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:44.783 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:52:44.784 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:44.786 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:52:44.786 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:44.787 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:52:44.787 DEBUG:teuthology.parallel:result is None 2023-11-28T11:52:44.788 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:52:44.861 INFO:tasks.ceph.mds.c.smithi144.stdout:starting mds.c at 2023-11-28T11:52:44.862 INFO:tasks.ceph.mds.a.smithi144.stdout:starting mds.a at 2023-11-28T11:52:44.875 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T11:52:44.875 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T11:52:44.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.915+0000 7f6053db0700 1 Processor -- start 2023-11-28T11:52:44.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.916+0000 7f6053db0700 1 -- start start 2023-11-28T11:52:44.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.916+0000 7f6053db0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f604c161340 0x7f604c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.916+0000 7f6053db0700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c0d1950 con 0x7f604c15c9b0 2023-11-28T11:52:44.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6053db0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f604c0b5520 con 0x7f604c0d2320 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6053db0700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604c0b5800 con 0x7f604c161340 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f605134b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f604c161340 0x7f604c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6051b4c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f604c15c9b0 0x7f604c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53830/0 (socket says 172.21.15.144:53830) 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6051b4c700 1 -- 172.21.15.144:0/2432744673 learned_addr learned my addr 172.21.15.144:0/2432744673 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f605134b700 1 -- 172.21.15.144:0/2432744673 >> v1:172.21.15.177:6789/0 conn(0x7f604c0d2320 legacy=0x7f604c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f605134b700 1 -- 172.21.15.144:0/2432744673 >> v1:172.21.15.144:6789/0 conn(0x7f604c15c9b0 legacy=0x7f604c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f605134b700 1 -- 172.21.15.144:0/2432744673 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0d1300 con 0x7f604c161340 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f605134b700 1 --2- 172.21.15.144:0/2432744673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c0d1de0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6048009840 tx=0x7f6048009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa18cefbdd4ac11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6050b4a700 1 -- 172.21.15.144:0/2432744673 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048004230 con 0x7f604c161340 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6050b4a700 1 -- 172.21.15.144:0/2432744673 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048004390 con 0x7f604c161340 2023-11-28T11:52:44.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.917+0000 7f6050b4a700 1 -- 172.21.15.144:0/2432744673 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60480036a0 con 0x7f604c161340 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 -- 172.21.15.144:0/2432744673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 msgr2=0x7f604c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 --2- 172.21.15.144:0/2432744673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c0d1de0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6048009840 tx=0x7f6048009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 -- 172.21.15.144:0/2432744673 shutdown_connections 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 --2- 172.21.15.144:0/2432744673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c0d1de0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 -- 172.21.15.144:0/2432744673 >> 172.21.15.144:0/2432744673 conn(0x7f604c072600 msgr2=0x7f604c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 -- 172.21.15.144:0/2432744673 shutdown_connections 2023-11-28T11:52:44.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.918+0000 7f6053db0700 1 -- 172.21.15.144:0/2432744673 wait complete. 2023-11-28T11:52:44.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.919+0000 7f6053db0700 1 Processor -- start 2023-11-28T11:52:44.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.919+0000 7f6053db0700 1 -- start start 2023-11-28T11:52:44.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f604c0d2320 0x7f604c0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f604c15c9b0 0x7f604c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.923 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c1e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c150e90 con 0x7f604c161340 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c0c8830 con 0x7f604c0d2320 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c1565a0 con 0x7f604c15c9b0 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c1e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c1e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37128/0 (socket says 172.21.15.144:37128) 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 -- 172.21.15.144:0/2515086677 learned_addr learned my addr 172.21.15.144:0/2515086677 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6051b4c700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f604c0d2320 0x7f604c0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 -- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f604c15c9b0 msgr2=0x7f604c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f604c15c9b0 0x7f604c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 -- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f604c0d2320 msgr2=0x7f604c0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f604c0d2320 0x7f604c0c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 -- 172.21.15.144:0/2515086677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60480094a0 con 0x7f604c161340 2023-11-28T11:52:44.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f605234d700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c1e6460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f604400ba10 tx=0x7f604400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400c9b0 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604400cb50 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044014440 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c15cd80 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.920+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c0d15e0 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.921+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c15cf10 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.922+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f604401b030 con 0x7f604c161340 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.922+0000 7f60427fc700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f60300682f0 0x7f603006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.922+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f60440854a0 con 0x7f604c161340 2023-11-28T11:52:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.922+0000 7f6051b4c700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f60300682f0 0x7f603006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.923+0000 7f6051b4c700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f60300682f0 0x7f603006a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f604c0b91d0 tx=0x7f603c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:44.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:44.926+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f604404d5e0 con 0x7f604c161340 2023-11-28T11:52:45.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:45.129+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f604c0745c0 con 0x7f604c161340 2023-11-28T11:52:46.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.132+0000 7f60427fc700 1 -- 172.21.15.144:0/2515086677 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6044052030 con 0x7f604c161340 2023-11-28T11:52:46.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f60300682f0 msgr2=0x7f603006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f60300682f0 0x7f603006a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f604c0b91d0 tx=0x7f603c009040 comp rx=0 tx=0).stop 2023-11-28T11:52:46.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 msgr2=0x7f604c1e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c1e6460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f604400ba10 tx=0x7f604400bdd0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 shutdown_connections 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604c161340 0x7f604c1e6460 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f60300682f0 0x7f603006a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f604c0d2320 0x7f604c0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 --2- 172.21.15.144:0/2515086677 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f604c15c9b0 0x7f604c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 >> 172.21.15.144:0/2515086677 conn(0x7f604c072600 msgr2=0x7f604c0b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 shutdown_connections 2023-11-28T11:52:46.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.135+0000 7f6053db0700 1 -- 172.21.15.144:0/2515086677 wait complete. 2023-11-28T11:52:46.139 INFO:teuthology.orchestra.run.smithi144.stderr: removed all blocklist entries 2023-11-28T11:52:46.151 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:52:46.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.275+0000 7f9a100d2700 1 Processor -- start 2023-11-28T11:52:46.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.276+0000 7f9a100d2700 1 -- start start 2023-11-28T11:52:46.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.276+0000 7f9a100d2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a0815cef0 0x7f9a080af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.276+0000 7f9a100d2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a080cbdf0 0x7f9a0806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.276+0000 7f9a100d2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a080cc5f0 con 0x7f9a0806b8c0 2023-11-28T11:52:46.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.276+0000 7f9a100d2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0806bd40 con 0x7f9a0815cef0 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.276+0000 7f9a100d2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0815d470 con 0x7f9a080cbdf0 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a0815cef0 0x7f9a080af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a0815cef0 0x7f9a080af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37136/0 (socket says 172.21.15.144:37136) 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 -- 172.21.15.144:0/4162584424 learned_addr learned my addr 172.21.15.144:0/4162584424 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0d66d700 1 --2- 172.21.15.144:0/4162584424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a080cbdf0 0x7f9a0806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 -- 172.21.15.144:0/4162584424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a080cbdf0 msgr2=0x7f9a0806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 --2- 172.21.15.144:0/4162584424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a080cbdf0 0x7f9a0806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 -- 172.21.15.144:0/4162584424 >> v1:172.21.15.177:6789/0 conn(0x7f9a0806b8c0 legacy=0x7f9a08071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 -- 172.21.15.144:0/4162584424 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0815d750 con 0x7f9a0815cef0 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0de6e700 1 --2- 172.21.15.144:0/4162584424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0815cef0 0x7f9a080af400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f99f80098f0 tx=0x7f99f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109196c6ca3dbd2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0ce6c700 1 -- 172.21.15.144:0/4162584424 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800b8a0 con 0x7f9a0815cef0 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0ce6c700 1 -- 172.21.15.144:0/4162584424 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800ba00 con 0x7f9a0815cef0 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.277+0000 7f9a0ce6c700 1 -- 172.21.15.144:0/4162584424 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800bd90 con 0x7f9a0815cef0 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 -- 172.21.15.144:0/4162584424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0815cef0 msgr2=0x7f9a080af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 --2- 172.21.15.144:0/4162584424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0815cef0 0x7f9a080af400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f99f80098f0 tx=0x7f99f8009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 -- 172.21.15.144:0/4162584424 shutdown_connections 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 --2- 172.21.15.144:0/4162584424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0815cef0 0x7f9a080af400 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 --2- 172.21.15.144:0/4162584424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a080cbdf0 0x7f9a0806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 -- 172.21.15.144:0/4162584424 >> 172.21.15.144:0/4162584424 conn(0x7f9a080d0000 msgr2=0x7f9a080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:46.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 -- 172.21.15.144:0/4162584424 shutdown_connections 2023-11-28T11:52:46.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.278+0000 7f9a100d2700 1 -- 172.21.15.144:0/4162584424 wait complete. 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.279+0000 7f9a100d2700 1 Processor -- start 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.279+0000 7f9a100d2700 1 -- start start 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 0x7f9a080c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a080cbdf0 0x7f9a080b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a0815cef0 0x7f9a080c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b5530 con 0x7f9a0806b8c0 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b4920 con 0x7f9a080cbdf0 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b4c00 con 0x7f9a0815cef0 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 0x7f9a080c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 0x7f9a080c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37142/0 (socket says 172.21.15.144:37142) 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 -- 172.21.15.144:0/3802896435 learned_addr learned my addr 172.21.15.144:0/3802896435 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0d66d700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a080cbdf0 0x7f9a080b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 -- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a0815cef0 msgr2=0x7f9a080c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0e66f700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a0815cef0 0x7f9a080c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a0815cef0 0x7f9a080c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 -- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a080cbdf0 msgr2=0x7f9a080b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a080cbdf0 0x7f9a080b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 -- 172.21.15.144:0/3802896435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8009550 con 0x7f9a0806b8c0 2023-11-28T11:52:46.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0d66d700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a080cbdf0 0x7f9a080b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0e66f700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a0815cef0 0x7f9a080c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a0de6e700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 0x7f9a080c8c70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009c40 tx=0x7f99f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80036a0 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f801e070 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.280+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0815dd10 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.281+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8003a40 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.281+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0815da30 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.281+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0815dff0 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.282+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99f800f050 con 0x7f9a0806b8c0 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.282+0000 7f99fe7fc700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f99ec0682f0 0x7f99ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.282+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f99f8020070 con 0x7f9a0806b8c0 2023-11-28T11:52:46.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.282+0000 7f9a0d66d700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f99ec0682f0 0x7f99ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.283+0000 7f9a0d66d700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f99ec0682f0 0x7f99ec06a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a0806c340 tx=0x7f9a04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:46.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.287+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99f8053ae0 con 0x7f9a0806b8c0 2023-11-28T11:52:46.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.541+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9a080b5810 con 0x7f9a0806b8c0 2023-11-28T11:52:46.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.541+0000 7f99fe7fc700 1 -- 172.21.15.144:0/3802896435 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f99f8057100 con 0x7f9a0806b8c0 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "auth_dump": [ 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.a", 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlzuwFOBAAtN3Mhkt5bwjiPFumLWf5LQ==", 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.544 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.b", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlmkasCBAAaYU+0cOXvyDnawSPLqNQBg==", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:46.545 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.c", 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlyo2xAxAARq98KYTF0ViGGgRsgbiHJg==", 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.546 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.d", 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVltR3sDxAAkRdNJ3ar7Vh/OwKimeVIGQ==", 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.0", 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAE1GVlRYZuNBAAkRLYVWzVGa/wVN7KApc9pA==", 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.1", 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAH1GVlzE4mNBAA8fJmDI1X35Kb7ffo7fKFGQ==", 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.2", 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAK1GVl0BB5OBAAJXjaD/H42vlurqW5ReH8QQ==", 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.549 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.3", 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAN1GVllXIyOBAAHYoY70u/C1a0B/IZhBZ3kw==", 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.550 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.4", 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAU1GVlWMw2ARAAQDzOPIY+tj6nPKoQ5d1Opw==", 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.5", 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAX1GVlwfFcABAAzJkyTNckY6Ua/4KuHrTn2A==", 2023-11-28T11:52:46.551 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.6", 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAa1GVldpcCABAAY7IyYyuR3TMLzueAnd70vA==", 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.552 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.7", 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAc1GVlnehWOxAA+UrMHeuJP4Btd7AlMXMyYA==", 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.553 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.0", 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVl4NRbFBAA6tNlYbw6YiCQ4X7ewAOxAw==", 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow r", 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.1", 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlfaJnHRAA4AdUVgWb3JW1D50Vj+quDQ==", 2023-11-28T11:52:46.554 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.2", 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlxfcVGhAAEX5yipIE6u3NvW4h8fLhjw==", 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.555 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.3", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlF0KoIBAAewnQNQ6UxhSPTRCTU/mgtQ==", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:52:46.556 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.admin", 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlkOuRGxAAVRU3tb5dFIW29oanM4hW6g==", 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.557 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mgr.x", 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVl+4ibNBAAmnsKW1HqPNEAHCtefjHz4A==", 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile mgr", 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mgr.y", 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlU3zUMBAAYSJLQSW0CAYPWfcdqxqHbA==", 2023-11-28T11:52:46.558 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile mgr", 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2023-11-28T11:52:46.559 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f99ec0682f0 msgr2=0x7f99ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f99ec0682f0 0x7f99ec06a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a0806c340 tx=0x7f9a04008040 comp rx=0 tx=0).stop 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 msgr2=0x7f9a080c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 0x7f9a080c8c70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009c40 tx=0x7f99f8004750 comp rx=0 tx=0).stop 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 shutdown_connections 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0806b8c0 0x7f9a080c8c70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f99ec0682f0 0x7f99ec06a7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a0815cef0 0x7f9a080c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 --2- 172.21.15.144:0/3802896435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a080cbdf0 0x7f9a080b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 >> 172.21.15.144:0/3802896435 conn(0x7f9a080d0000 msgr2=0x7f9a0806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:46.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 shutdown_connections 2023-11-28T11:52:46.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.544+0000 7f9a100d2700 1 -- 172.21.15.144:0/3802896435 wait complete. 2023-11-28T11:52:46.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:46.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.681+0000 7faba1cc9700 1 Processor -- start 2023-11-28T11:52:46.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.681+0000 7faba1cc9700 1 -- start start 2023-11-28T11:52:46.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7faba1cc9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab9c0cb620 0x7fab9c0ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7faba1cc9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fab9c0cf070 0x7fab9c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7faba1cc9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0cf4d0 con 0x7fab9c0b3090 2023-11-28T11:52:46.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7faba1cc9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c0ccbb0 con 0x7fab9c0cb620 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7faba1cc9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c15ad60 con 0x7fab9c0cf070 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab9c0cb620 0x7fab9c0ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fab9c0b3090 0x7fab9c0cc4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53862/0 (socket says 172.21.15.144:53862) 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9b7fe700 1 -- 172.21.15.144:0/472595541 learned_addr learned my addr 172.21.15.144:0/472595541 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9affd700 1 -- 172.21.15.144:0/472595541 >> v2:172.21.15.177:3300/0 conn(0x7fab9c0cf070 msgr2=0x7fab9c0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9affd700 1 --2- 172.21.15.144:0/472595541 >> v2:172.21.15.177:3300/0 conn(0x7fab9c0cf070 0x7fab9c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9affd700 1 -- 172.21.15.144:0/472595541 >> v1:172.21.15.144:6789/0 conn(0x7fab9c0b3090 legacy=0x7fab9c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9a7fc700 1 -- 172.21.15.144:0/472595541 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202222269 0 0) 0x7fab9c0cf4d0 con 0x7fab9c0b3090 2023-11-28T11:52:46.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9affd700 1 -- 172.21.15.144:0/472595541 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0adb90 con 0x7fab9c0cb620 2023-11-28T11:52:46.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9affd700 1 --2- 172.21.15.144:0/472595541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cb620 0x7fab9c0ceb30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab9c071dc0 tx=0x7fab8c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340b5fbbdd479907 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:46.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9a7fc700 1 -- 172.21.15.144:0/472595541 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c011830 con 0x7fab9c0cb620 2023-11-28T11:52:46.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9a7fc700 1 -- 172.21.15.144:0/472595541 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c011990 con 0x7fab9c0cb620 2023-11-28T11:52:46.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.682+0000 7fab9a7fc700 1 -- 172.21.15.144:0/472595541 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c011d20 con 0x7fab9c0cb620 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 -- 172.21.15.144:0/472595541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cb620 msgr2=0x7fab9c0ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 --2- 172.21.15.144:0/472595541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cb620 0x7fab9c0ceb30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fab9c071dc0 tx=0x7fab8c009b20 comp rx=0 tx=0).stop 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 -- 172.21.15.144:0/472595541 shutdown_connections 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 --2- 172.21.15.144:0/472595541 >> v2:172.21.15.177:3300/0 conn(0x7fab9c0cf070 0x7fab9c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 --2- 172.21.15.144:0/472595541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cb620 0x7fab9c0ceb30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 -- 172.21.15.144:0/472595541 >> 172.21.15.144:0/472595541 conn(0x7fab9c09a230 msgr2=0x7fab9c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 -- 172.21.15.144:0/472595541 shutdown_connections 2023-11-28T11:52:46.688 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.683+0000 7faba1cc9700 1 -- 172.21.15.144:0/472595541 wait complete. 2023-11-28T11:52:46.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.684+0000 7faba1cc9700 1 Processor -- start 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.684+0000 7faba1cc9700 1 -- start start 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7faba1cc9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab9c0b3090 0x7fab9c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7faba1cc9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab9c0cb620 0x7fab9c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7faba1cc9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 0x7fab9c1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7faba1cc9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c06d3c0 con 0x7fab9c0cf070 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7faba1cc9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1f5cd0 con 0x7fab9c0b3090 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7faba1cc9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c156e10 con 0x7fab9c0cb620 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 0x7fab9c1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7fab9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 0x7fab9c1f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37160/0 (socket says 172.21.15.144:37160) 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7fab9bfff700 1 -- 172.21.15.144:0/959423820 learned_addr learned my addr 172.21.15.144:0/959423820 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7fab9b7fe700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab9c0b3090 0x7fab9c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.685+0000 7fab9affd700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab9c0cb620 0x7fab9c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab9bfff700 1 -- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab9c0cb620 msgr2=0x7fab9c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab9bfff700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab9c0cb620 0x7fab9c06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab9bfff700 1 -- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab9c0b3090 msgr2=0x7fab9c158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab9bfff700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab9c0b3090 0x7fab9c158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab9bfff700 1 -- 172.21.15.144:0/959423820 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c0094a0 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab9bfff700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 0x7fab9c1f5790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fab9400b970 tx=0x7fab9400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400c9b0 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9400cb30 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0cd0b0 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94014440 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.686+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0ccdd0 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.687+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c0cd390 con 0x7fab9c0cf070 2023-11-28T11:52:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.687+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab9401b030 con 0x7fab9c0cf070 2023-11-28T11:52:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.688+0000 7fab83fff700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fab880682f0 0x7fab8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.688+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fab940855b0 con 0x7fab9c0cf070 2023-11-28T11:52:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.688+0000 7fab9b7fe700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fab880682f0 0x7fab8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.688+0000 7fab9b7fe700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fab880682f0 0x7fab8806a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fab90000f60 tx=0x7fab90007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:46.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.692+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab94052160 con 0x7fab9c0cf070 2023-11-28T11:52:46.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.876+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab9c156f80 con 0x7fab9c0cf070 2023-11-28T11:52:46.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.877+0000 7fab83fff700 1 -- 172.21.15.144:0/959423820 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14257 (secure 0 0 0) 0x7fab94019090 con 0x7fab9c0cf070 2023-11-28T11:52:46.879 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:46.879 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":42,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:46.125055+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:46.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fab880682f0 msgr2=0x7fab8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fab880682f0 0x7fab8806a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fab90000f60 tx=0x7fab90007be0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 msgr2=0x7fab9c1f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:46.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 0x7fab9c1f5790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fab9400b970 tx=0x7fab9400bd30 comp rx=0 tx=0).stop 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 shutdown_connections 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab9c0cf070 0x7fab9c1f5790 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fab880682f0 0x7fab8806a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab9c0b3090 0x7fab9c158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 --2- 172.21.15.144:0/959423820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab9c0cb620 0x7fab9c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 >> 172.21.15.144:0/959423820 conn(0x7fab9c09a230 msgr2=0x7fab9c0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 shutdown_connections 2023-11-28T11:52:46.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:46.879+0000 7faba1cc9700 1 -- 172.21.15.144:0/959423820 wait complete. 2023-11-28T11:52:46.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:52:46.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:52:47.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.015+0000 7f87c535f700 1 Processor -- start 2023-11-28T11:52:47.017 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.015+0000 7f87c535f700 1 -- start start 2023-11-28T11:52:47.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87c535f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 0x7f87c00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87c535f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c00d1950 con 0x7f87c0161340 2023-11-28T11:52:47.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87c535f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c00d24f0 con 0x7f87c00cd7c0 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87c535f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00b5800 con 0x7f87c015c9b0 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87be7fc700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f87c0161340 0x7f87c00d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53876/0 (socket says 172.21.15.144:53876) 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87be7fc700 1 -- 172.21.15.144:0/3624772887 learned_addr learned my addr 172.21.15.144:0/3624772887 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87beffd700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 0x7f87c00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908882815 0 0) 0x7f87c00d1950 con 0x7f87c0161340 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a80035b0 con 0x7f87c0161340 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165995852 0 0) 0x7f87c00d24f0 con 0x7f87c00cd7c0 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c00d1950 con 0x7f87c00cd7c0 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189703485 0 0) 0x7f87a80035b0 con 0x7f87c0161340 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 msgr2=0x7f87c00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 0x7f87c00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 >> v1:172.21.15.177:6789/0 conn(0x7f87c00cd7c0 legacy=0x7f87c00b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00b5520 con 0x7f87c0161340 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f87b0001670 con 0x7f87c0161340 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 >> v1:172.21.15.144:6789/0 conn(0x7f87c0161340 legacy=0x7f87c00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87a8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.016+0000 7f87bdffb700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a8004850 0x7f87a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 0x7f87a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00b5800 con 0x7f87a8003bc0 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a800baf0 con 0x7f87a8004850 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a800bcb0 con 0x7f87c015c9b0 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87a8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87be7fc700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a8004850 0x7f87a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bf7fe700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 0x7f87a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 -- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 msgr2=0x7f87a8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015c9b0 0x7f87a8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 -- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a8004850 msgr2=0x7f87a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a8004850 0x7f87a8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 -- 172.21.15.144:0/3624772887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80035b0 con 0x7f87a8003bc0 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87be7fc700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87a8004850 0x7f87a8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87beffd700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87a8003fb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009b60 tx=0x7f87a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7a3067cc1a059c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e6c0 con 0x7f87a8003bc0 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b400e820 con 0x7f87a8003bc0 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.017+0000 7f87bdffb700 1 -- 172.21.15.144:0/3624772887 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400eb90 con 0x7f87a8003bc0 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87c535f700 1 -- 172.21.15.144:0/3624772887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 msgr2=0x7f87a8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87c535f700 1 --2- 172.21.15.144:0/3624772887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87a8003fb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009b60 tx=0x7f87a8005880 comp rx=0 tx=0).stop 2023-11-28T11:52:47.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87bf7fe700 1 -- 172.21.15.144:0/3624772887 reap_dead start 2023-11-28T11:52:47.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87c535f700 1 -- 172.21.15.144:0/3624772887 shutdown_connections 2023-11-28T11:52:47.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87c535f700 1 -- 172.21.15.144:0/3624772887 >> 172.21.15.144:0/3624772887 conn(0x7f87c0072600 msgr2=0x7f87c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:47.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87c535f700 1 -- 172.21.15.144:0/3624772887 shutdown_connections 2023-11-28T11:52:47.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.018+0000 7f87c535f700 1 -- 172.21.15.144:0/3624772887 wait complete. 2023-11-28T11:52:47.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.019+0000 7f87c535f700 1 Processor -- start 2023-11-28T11:52:47.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.019+0000 7f87c535f700 1 -- start start 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87c535f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87c00c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87c535f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015d430 0x7f87c015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87c535f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87c0146130 0x7f87c01465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87c535f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c015dde0 con 0x7f87a8003bc0 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87c535f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0144080 con 0x7f87c0146130 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87c535f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c00caa90 con 0x7f87c015d430 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87c00c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87c00c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37174/0 (socket says 172.21.15.144:37174) 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 -- 172.21.15.144:0/2406498503 learned_addr learned my addr 172.21.15.144:0/2406498503 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87be7fc700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015d430 0x7f87c015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87bf7fe700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87c0146130 0x7f87c01465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 -- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015d430 msgr2=0x7f87c015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015d430 0x7f87c015d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 -- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87c0146130 msgr2=0x7f87c01465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87c0146130 0x7f87c01465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87beffd700 1 -- 172.21.15.144:0/2406498503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c00d15e0 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.020+0000 7f87bf7fe700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87c0146130 0x7f87c01465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.021+0000 7f87beffd700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87c00c5e60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f87b40042f0 tx=0x7f87b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.021+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e6c0 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.021+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4003ab0 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.021+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c00d1300 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.021+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4003e60 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.021+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c015cd80 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.022+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c00745c0 con 0x7f87a8003bc0 2023-11-28T11:52:47.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.022+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87b4025070 con 0x7f87a8003bc0 2023-11-28T11:52:47.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.022+0000 7f87a77fe700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f87ac0682f0 0x7f87ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.023+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f87b4087310 con 0x7f87a8003bc0 2023-11-28T11:52:47.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.023+0000 7f87be7fc700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f87ac0682f0 0x7f87ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.023+0000 7f87be7fc700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f87ac0682f0 0x7f87ac06a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87b0001200 tx=0x7f87b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.027+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87b4053dd0 con 0x7f87a8003bc0 2023-11-28T11:52:47.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.211+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87c00d08a0 con 0x7f87a8003bc0 2023-11-28T11:52:47.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.211+0000 7f87a77fe700 1 -- 172.21.15.144:0/2406498503 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87b40573f0 con 0x7f87a8003bc0 2023-11-28T11:52:47.213 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f87ac0682f0 msgr2=0x7f87ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f87ac0682f0 0x7f87ac06a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87b0001200 tx=0x7f87b0009040 comp rx=0 tx=0).stop 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 msgr2=0x7f87c00c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87c00c5e60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f87b40042f0 tx=0x7f87b40047c0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 shutdown_connections 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87a8003bc0 0x7f87c00c5e60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f87ac0682f0 0x7f87ac06a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87c0146130 0x7f87c01465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 --2- 172.21.15.144:0/2406498503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87c015d430 0x7f87c015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 >> 172.21.15.144:0/2406498503 conn(0x7f87c0072600 msgr2=0x7f87c00b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:47.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 shutdown_connections 2023-11-28T11:52:47.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.214+0000 7f87c535f700 1 -- 172.21.15.144:0/2406498503 wait complete. 2023-11-28T11:52:47.228 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:52:47.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.349+0000 7f109b74c700 1 Processor -- start 2023-11-28T11:52:47.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.349+0000 7f109b74c700 1 -- start start 2023-11-28T11:52:47.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f109b74c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1094071040 0x7f1094071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f109b74c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f109406dee0 0x7f109406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f109b74c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1094071950 con 0x7f109406cdc0 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f109b74c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1094161550 con 0x7f109406dee0 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f109b74c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109415d060 con 0x7f1094071040 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f109406dee0 0x7f109406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f109406dee0 0x7f109406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37182/0 (socket says 172.21.15.144:37182) 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 -- 172.21.15.144:0/3847017235 learned_addr learned my addr 172.21.15.144:0/3847017235 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 -- 172.21.15.144:0/3847017235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1094071040 msgr2=0x7f1094071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 --2- 172.21.15.144:0/3847017235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1094071040 0x7f1094071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 -- 172.21.15.144:0/3847017235 >> v1:172.21.15.177:6789/0 conn(0x7f109406cdc0 legacy=0x7f109406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.350+0000 7f1098ce7700 1 -- 172.21.15.144:0/3847017235 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109415d340 con 0x7f109406dee0 2023-11-28T11:52:47.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f1098ce7700 1 --2- 172.21.15.144:0/3847017235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f109406e350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f10900098f0 tx=0x7f1090009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f7260ba7ea91a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f108bfff700 1 -- 172.21.15.144:0/3847017235 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090004230 con 0x7f109406dee0 2023-11-28T11:52:47.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f108bfff700 1 -- 172.21.15.144:0/3847017235 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090004390 con 0x7f109406dee0 2023-11-28T11:52:47.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f108bfff700 1 -- 172.21.15.144:0/3847017235 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10900036a0 con 0x7f109406dee0 2023-11-28T11:52:47.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 -- 172.21.15.144:0/3847017235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 msgr2=0x7f109406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 --2- 172.21.15.144:0/3847017235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f109406e350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f10900098f0 tx=0x7f1090009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:47.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 -- 172.21.15.144:0/3847017235 shutdown_connections 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 --2- 172.21.15.144:0/3847017235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1094071040 0x7f1094071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 --2- 172.21.15.144:0/3847017235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f109406e350 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 -- 172.21.15.144:0/3847017235 >> 172.21.15.144:0/3847017235 conn(0x7f10940d0000 msgr2=0x7f10940d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 -- 172.21.15.144:0/3847017235 shutdown_connections 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.351+0000 7f109b74c700 1 -- 172.21.15.144:0/3847017235 wait complete. 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 Processor -- start 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 -- start start 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109406cdc0 0x7f109415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f10940acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1094071040 0x7f10940ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940ae820 con 0x7f109406dee0 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109415b4f0 con 0x7f1094071040 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f109b74c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940adba0 con 0x7f109406cdc0 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f1098ce7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f10940acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f1098ce7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f10940acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37188/0 (socket says 172.21.15.144:37188) 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.353+0000 7f1098ce7700 1 -- 172.21.15.144:0/2737519796 learned_addr learned my addr 172.21.15.144:0/2737519796 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f10994e8700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109406cdc0 0x7f109415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1099ce9700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1094071040 0x7f10940ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1098ce7700 1 -- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109406cdc0 msgr2=0x7f109415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1098ce7700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109406cdc0 0x7f109415ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1098ce7700 1 -- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1094071040 msgr2=0x7f10940ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1098ce7700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1094071040 0x7f10940ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1098ce7700 1 -- 172.21.15.144:0/2737519796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090009550 con 0x7f109406dee0 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1099ce9700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1094071040 0x7f10940ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1098ce7700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f10940acfe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f109406c7b0 tx=0x7f1090004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000b660 con 0x7f109406dee0 2023-11-28T11:52:47.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000b7c0 con 0x7f109406dee0 2023-11-28T11:52:47.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000bb50 con 0x7f109406dee0 2023-11-28T11:52:47.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109415d900 con 0x7f109406dee0 2023-11-28T11:52:47.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.354+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109415d620 con 0x7f109406dee0 2023-11-28T11:52:47.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.355+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109406e890 con 0x7f109406dee0 2023-11-28T11:52:47.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.356+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1090017020 con 0x7f109406dee0 2023-11-28T11:52:47.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.356+0000 7f1089ffb700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10780682f0 0x7f107806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.356+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f10900871c0 con 0x7f109406dee0 2023-11-28T11:52:47.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.356+0000 7f10994e8700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10780682f0 0x7f107806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.361 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.356+0000 7f10994e8700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10780682f0 0x7f107806a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1084009970 tx=0x7f1084009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.362 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.360+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1090053d70 con 0x7f109406dee0 2023-11-28T11:52:47.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.542+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10940ae960 con 0x7f109406dee0 2023-11-28T11:52:47.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.542+0000 7f1089ffb700 1 -- 172.21.15.144:0/2737519796 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10940ae960 con 0x7f109406dee0 2023-11-28T11:52:47.544 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:52:47.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10780682f0 msgr2=0x7f107806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10780682f0 0x7f107806a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1084009970 tx=0x7f1084009110 comp rx=0 tx=0).stop 2023-11-28T11:52:47.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 msgr2=0x7f10940acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f10940acfe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f109406c7b0 tx=0x7f1090004750 comp rx=0 tx=0).stop 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 shutdown_connections 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10780682f0 0x7f107806a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1094071040 0x7f10940ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f109406cdc0 0x7f109415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 --2- 172.21.15.144:0/2737519796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f109406dee0 0x7f10940acfe0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 >> 172.21.15.144:0/2737519796 conn(0x7f10940d0000 msgr2=0x7f109415bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 shutdown_connections 2023-11-28T11:52:47.547 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.544+0000 7f109b74c700 1 -- 172.21.15.144:0/2737519796 wait complete. 2023-11-28T11:52:47.558 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:52:47.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.677+0000 7f7cd7238700 1 Processor -- start 2023-11-28T11:52:47.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.677+0000 7f7cd7238700 1 -- start start 2023-11-28T11:52:47.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd7238700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7cd006ba90 0x7f7cd00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd7238700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cd00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd7238700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd00cc5f0 con 0x7f7cd015cef0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd7238700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd015d470 con 0x7f7cd006ba90 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd7238700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd015d750 con 0x7f7cd00cbdf0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd4fd4700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7cd015cef0 0x7f7cd006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53908/0 (socket says 172.21.15.144:53908) 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd4fd4700 1 -- 172.21.15.144:0/3391449390 learned_addr learned my addr 172.21.15.144:0/3391449390 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737998282 0 0) 0x7f7cd00cc5f0 con 0x7f7cd015cef0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc00035b0 con 0x7f7cd015cef0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccffff700 1 --2- 172.21.15.144:0/3391449390 >> v2:172.21.15.177:3300/0 conn(0x7f7cd006ba90 0x7f7cd00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd57d5700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cd00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 92813915 0 0) 0x7f7cc00035b0 con 0x7f7cd015cef0 2023-11-28T11:52:47.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 msgr2=0x7f7cd00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cd00ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 >> v2:172.21.15.177:3300/0 conn(0x7f7cd006ba90 msgr2=0x7f7cd00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 --2- 172.21.15.144:0/3391449390 >> v2:172.21.15.177:3300/0 conn(0x7f7cd006ba90 0x7f7cd00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd015da30 con 0x7f7cd015cef0 2023-11-28T11:52:47.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f7cc4001670 con 0x7f7cd015cef0 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd57d5700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cd00ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 >> v1:172.21.15.144:6789/0 conn(0x7f7cd015cef0 legacy=0x7f7cd006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cc0003a00 0x7f7cc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cc0004850 0x7f7cc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cc0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd015d470 con 0x7f7cc0003a00 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd015d750 con 0x7f7cc0004850 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc000ba60 con 0x7f7cd00cbdf0 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.678+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cc0003a00 0x7f7cc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7ccffff700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cc0004850 0x7f7cc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd57d5700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cc0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd4fd4700 1 -- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 msgr2=0x7f7cc0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cc0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd4fd4700 1 -- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cc0004850 msgr2=0x7f7cc0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cc0004850 0x7f7cc0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd4fd4700 1 -- 172.21.15.144:0/3391449390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00035b0 con 0x7f7cc0003a00 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd57d5700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cd00cbdf0 0x7f7cc0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7ccffff700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cc0004850 0x7f7cc0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cc0003a00 0x7f7cc0003e10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c350 tx=0x7f7cc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600b4a900a6ce863 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400ce50 con 0x7f7cc0003a00 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc4005960 con 0x7f7cc0003a00 2023-11-28T11:52:47.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.679+0000 7f7ccf7fe700 1 -- 172.21.15.144:0/3391449390 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4005d10 con 0x7f7cc0003a00 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd7238700 1 -- 172.21.15.144:0/3391449390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cc0003a00 msgr2=0x7f7cc0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd7238700 1 --2- 172.21.15.144:0/3391449390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cc0003a00 0x7f7cc0003e10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c350 tx=0x7f7cc0005880 comp rx=0 tx=0).stop 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd57d5700 1 -- 172.21.15.144:0/3391449390 reap_dead start 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd7238700 1 -- 172.21.15.144:0/3391449390 shutdown_connections 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd7238700 1 -- 172.21.15.144:0/3391449390 >> 172.21.15.144:0/3391449390 conn(0x7f7cd00d0000 msgr2=0x7f7cd00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd7238700 1 -- 172.21.15.144:0/3391449390 shutdown_connections 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.680+0000 7f7cd7238700 1 -- 172.21.15.144:0/3391449390 wait complete. 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.681+0000 7f7cd7238700 1 Processor -- start 2023-11-28T11:52:47.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.681+0000 7f7cd7238700 1 -- start start 2023-11-28T11:52:47.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cc0003a00 0x7f7cd0145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 0x7f7cd00c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cd00b2e30 0x7f7cd00b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00b37e0 con 0x7f7cd00c4b60 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00c6230 con 0x7f7cd00b2e30 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd01f0480 con 0x7f7cc0003a00 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 0x7f7cd00c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 0x7f7cd00c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37210/0 (socket says 172.21.15.144:37210) 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 -- 172.21.15.144:0/2086999847 learned_addr learned my addr 172.21.15.144:0/2086999847 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cc0003a00 0x7f7cd0145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd57d5700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cd00b2e30 0x7f7cd00b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 -- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cc0003a00 msgr2=0x7f7cd0145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cc0003a00 0x7f7cd0145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 -- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cd00b2e30 msgr2=0x7f7cd00b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cd00b2e30 0x7f7cd00b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 -- 172.21.15.144:0/2086999847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd015dff0 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccffff700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 0x7f7cd00c4fd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc010870 tx=0x7f7cbc010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc004230 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc00c050 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd015dd10 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.682+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0045b0 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.683+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd00ca650 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.684+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd00c6370 con 0x7f7cd00c4b60 2023-11-28T11:52:47.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.684+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7cbc01c020 con 0x7f7cd00c4b60 2023-11-28T11:52:47.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.684+0000 7f7ccd7fa700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7cb40682f0 0x7f7cb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:47.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.684+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7cbc086060 con 0x7f7cd00c4b60 2023-11-28T11:52:47.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.684+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7cb40682f0 0x7f7cb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:47.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.685+0000 7f7cd4fd4700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7cb40682f0 0x7f7cb406a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4008fd0 tx=0x7f7cc400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:47.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.689+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7cbc052c10 con 0x7f7cd00c4b60 2023-11-28T11:52:47.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.874+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7cd00d0e60 con 0x7f7cd00c4b60 2023-11-28T11:52:47.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.874+0000 7f7ccd7fa700 1 -- 172.21.15.144:0/2086999847 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7cbc056230 con 0x7f7cd00c4b60 2023-11-28T11:52:47.876 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout:[ 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 1, 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": "rbd" 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 2, 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": ".mgr" 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:52:47.877 INFO:teuthology.orchestra.run.smithi144.stdout:] 2023-11-28T11:52:47.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.876+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7cb40682f0 msgr2=0x7f7cb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.876+0000 7f7cd7238700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7cb40682f0 0x7f7cb406a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4008fd0 tx=0x7f7cc400a000 comp rx=0 tx=0).stop 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.876+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 msgr2=0x7f7cd00c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.876+0000 7f7cd7238700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 0x7f7cd00c4fd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc010870 tx=0x7f7cbc010c30 comp rx=0 tx=0).stop 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 shutdown_connections 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7cb40682f0 0x7f7cb406a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7cd00b2e30 0x7f7cd00b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7cc0003a00 0x7f7cd0145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 --2- 172.21.15.144:0/2086999847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cd00c4b60 0x7f7cd00c4fd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 >> 172.21.15.144:0/2086999847 conn(0x7f7cd00d0000 msgr2=0x7f7cd00cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 shutdown_connections 2023-11-28T11:52:47.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:47.877+0000 7f7cd7238700 1 -- 172.21.15.144:0/2086999847 wait complete. 2023-11-28T11:52:47.891 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:52:47.891 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:52:48.015 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.014+0000 7fcafb538700 1 Processor -- start 2023-11-28T11:52:48.016 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.014+0000 7fcafb538700 1 -- start start 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcafb538700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcaf4157160 0x7fcaf415a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcafb538700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf4157530 con 0x7fcaf40af8e0 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcafb538700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf415aa60 con 0x7fcaf40afd60 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcafb538700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf415deb0 con 0x7fcaf4157160 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaf8ad3700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcaf40af8e0 0x7fcaf415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53932/0 (socket says 172.21.15.144:53932) 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaf8ad3700 1 -- 172.21.15.144:0/2097548196 learned_addr learned my addr 172.21.15.144:0/2097548196 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaf92d4700 1 --2- 172.21.15.144:0/2097548196 >> v2:172.21.15.177:3300/0 conn(0x7fcaf4157160 0x7fcaf415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363483506 0 0) 0x7fcaf4157530 con 0x7fcaf40af8e0 2023-11-28T11:52:48.020 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae00035b0 con 0x7fcaf40af8e0 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075988223 0 0) 0x7fcae00035b0 con 0x7fcaf40af8e0 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 >> v2:172.21.15.177:3300/0 conn(0x7fcaf4157160 msgr2=0x7fcaf415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 --2- 172.21.15.144:0/2097548196 >> v2:172.21.15.177:3300/0 conn(0x7fcaf4157160 0x7fcaf415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 >> v1:172.21.15.177:6789/0 conn(0x7fcaf40afd60 legacy=0x7fcaf415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf40c9820 con 0x7fcaf40af8e0 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fcae40017f0 con 0x7fcaf40af8e0 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 >> v1:172.21.15.144:6789/0 conn(0x7fcaf40af8e0 legacy=0x7fcaf415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.015+0000 7fcaebfff700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae0003a00 0x7fcae0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0004850 0x7fcae0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0005200 0x7fcae000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf415deb0 con 0x7fcae0005200 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000c110 con 0x7fcae0003a00 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000c2d0 con 0x7fcae0004850 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0005200 0x7fcae000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf92d4700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae0003a00 0x7fcae0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf8ad3700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0004850 0x7fcae0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0004850 msgr2=0x7fcae0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0004850 0x7fcae0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae0003a00 msgr2=0x7fcae0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcae0003a00 0x7fcae0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/2097548196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00035b0 con 0x7fcae0005200 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/2097548196 reap_dead start 2023-11-28T11:52:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0005200 0x7fcae000ba10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c190 tx=0x7fcae0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e59c24a748db85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec001db0 con 0x7fcae0005200 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec00ccf0 con 0x7fcae0005200 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.016+0000 7fcaebfff700 1 -- 172.21.15.144:0/2097548196 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec003c40 con 0x7fcae0005200 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 -- 172.21.15.144:0/2097548196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0005200 msgr2=0x7fcae000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0005200 0x7fcae000ba10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00c190 tx=0x7fcae0005880 comp rx=0 tx=0).stop 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 -- 172.21.15.144:0/2097548196 shutdown_connections 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 --2- 172.21.15.144:0/2097548196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcae0005200 0x7fcae000ba10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 -- 172.21.15.144:0/2097548196 >> 172.21.15.144:0/2097548196 conn(0x7fcaf40b0860 msgr2=0x7fcaf40b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 -- 172.21.15.144:0/2097548196 shutdown_connections 2023-11-28T11:52:48.023 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.017+0000 7fcafb538700 1 -- 172.21.15.144:0/2097548196 wait complete. 2023-11-28T11:52:48.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.018+0000 7fcafb538700 1 Processor -- start 2023-11-28T11:52:48.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.018+0000 7fcafb538700 1 -- start start 2023-11-28T11:52:48.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0005200 0x7fcaf4147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaf4156350 0x7fcaf406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 0x7fcaf40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40af190 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4157530 con 0x7fcaf4156350 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40c72c0 con 0x7fcae0005200 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 0x7fcaf40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 0x7fcaf40aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37232/0 (socket says 172.21.15.144:37232) 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/4132958504 learned_addr learned my addr 172.21.15.144:0/4132958504 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/4132958504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0005200 msgr2=0x7fcaf4147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0005200 0x7fcaf4147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/4132958504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaf4156350 msgr2=0x7fcaf406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaf4156350 0x7fcaf406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 -- 172.21.15.144:0/4132958504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf40c9de0 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.027 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcaf9ad5700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 0x7fcaf40aec50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005040 tx=0x7fcaec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec005070 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec0051d0 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf40c9b00 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.019+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec005540 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.020+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf40ca0c0 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.021+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf404e620 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.021+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcaec010050 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.021+0000 7fcae9ffb700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcad80682f0 0x7fcad806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.021+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fcaec086c00 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.021+0000 7fcaf92d4700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcad80682f0 0x7fcad806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.022+0000 7fcaf92d4700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcad80682f0 0x7fcad806a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009ae0 tx=0x7fcaf00093c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:48.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.025+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcaec053750 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.209+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcaf40b1730 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.811+0000 7fcae9ffb700 1 -- 172.21.15.144:0/4132958504 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcaec016070 con 0x7fcaf40ae7d0 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcad80682f0 msgr2=0x7fcad806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcad80682f0 0x7fcad806a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009ae0 tx=0x7fcaf00093c0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 msgr2=0x7fcaf40aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 0x7fcaf40aec50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcaec005040 tx=0x7fcaec0014a0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 shutdown_connections 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40ae7d0 0x7fcaf40aec50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcad80682f0 0x7fcad806a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcae0005200 0x7fcaf4147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 --2- 172.21.15.144:0/4132958504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcaf4156350 0x7fcaf406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 >> 172.21.15.144:0/4132958504 conn(0x7fcaf40b0860 msgr2=0x7fcaf40cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:48.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 shutdown_connections 2023-11-28T11:52:48.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.816+0000 7fcafb538700 1 -- 172.21.15.144:0/4132958504 wait complete. 2023-11-28T11:52:48.819 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' created 2023-11-28T11:52:48.832 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:52:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.965+0000 7fd7897d6700 1 Processor -- start 2023-11-28T11:52:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.965+0000 7fd7897d6700 1 -- start start 2023-11-28T11:52:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7897d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd7840b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7897d6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd784161340 0x7fd7840d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7897d6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7840d1950 con 0x7fd7840d2320 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7897d6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7840b5800 con 0x7fd784161340 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7897d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840b5520 con 0x7fd78415c9b0 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7837fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd7840d2320 0x7fd7840cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53952/0 (socket says 172.21.15.144:53952) 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7837fe700 1 -- 172.21.15.144:0/903113364 learned_addr learned my addr 172.21.15.144:0/903113364 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd782ffd700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd7840b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7827fc700 1 --2- 172.21.15.144:0/903113364 >> v2:172.21.15.177:3300/0 conn(0x7fd784161340 0x7fd7840d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218177049 0 0) 0x7fd7840d1950 con 0x7fd7840d2320 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd76c0035b0 con 0x7fd7840d2320 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 314488704 0 0) 0x7fd76c0035b0 con 0x7fd7840d2320 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 msgr2=0x7fd7840b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd7840b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 >> v2:172.21.15.177:3300/0 conn(0x7fd784161340 msgr2=0x7fd7840d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 --2- 172.21.15.144:0/903113364 >> v2:172.21.15.177:3300/0 conn(0x7fd784161340 0x7fd7840d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840d1300 con 0x7fd7840d2320 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fd77c0019f0 con 0x7fd7840d2320 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd782ffd700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd7840b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd7827fc700 1 --2- 172.21.15.144:0/903113364 >> v2:172.21.15.177:3300/0 conn(0x7fd784161340 0x7fd7840d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.966+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 >> v1:172.21.15.144:6789/0 conn(0x7fd7840d2320 legacy=0x7fd7840cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd76c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd76c003f40 0x7fd76c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd76c004d90 0x7fd76c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840b5800 con 0x7fd76c004d90 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840b5520 con 0x7fd76c003f40 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c00ba10 con 0x7fd78415c9b0 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd76c004d90 0x7fd76c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd782ffd700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd76c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7827fc700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd76c003f40 0x7fd76c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 -- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 msgr2=0x7fd76c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd78415c9b0 0x7fd76c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 -- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd76c003f40 msgr2=0x7fd76c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd76c003f40 0x7fd76c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 -- 172.21.15.144:0/903113364 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780094a0 con 0x7fd76c004d90 2023-11-28T11:52:48.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd7837fe700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd76c004d90 0x7fd76c005200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00a860 tx=0x7fd76c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b52c17c7ef0abf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c012600 con 0x7fd76c004d90 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c012780 con 0x7fd76c004d90 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.967+0000 7fd781ffb700 1 -- 172.21.15.144:0/903113364 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c012b30 con 0x7fd76c004d90 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7897d6700 1 -- 172.21.15.144:0/903113364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd76c004d90 msgr2=0x7fd76c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7897d6700 1 --2- 172.21.15.144:0/903113364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd76c004d90 0x7fd76c005200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00a860 tx=0x7fd76c005880 comp rx=0 tx=0).stop 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7837fe700 1 -- 172.21.15.144:0/903113364 reap_dead start 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7897d6700 1 -- 172.21.15.144:0/903113364 shutdown_connections 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7897d6700 1 -- 172.21.15.144:0/903113364 >> 172.21.15.144:0/903113364 conn(0x7fd784072600 msgr2=0x7fd7840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:48.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7897d6700 1 -- 172.21.15.144:0/903113364 shutdown_connections 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.968+0000 7fd7897d6700 1 -- 172.21.15.144:0/903113364 wait complete. 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.970+0000 7fd7897d6700 1 Processor -- start 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.970+0000 7fd7897d6700 1 -- start start 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.970+0000 7fd7897d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7840c5f40 0x7fd784153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.970+0000 7fd7897d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 0x7fd7841521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7897d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd784159180 0x7fd7841595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7897d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784159b30 con 0x7fd784151b50 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7897d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784152a40 con 0x7fd784159180 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7897d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd784152d20 con 0x7fd7840c5f40 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 0x7fd7841521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 0x7fd7841521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37250/0 (socket says 172.21.15.144:37250) 2023-11-28T11:52:48.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 -- 172.21.15.144:0/3973335224 learned_addr learned my addr 172.21.15.144:0/3973335224 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd782ffd700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7840c5f40 0x7fd784153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7837fe700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd784159180 0x7fd7841595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 -- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7840c5f40 msgr2=0x7fd784153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7840c5f40 0x7fd784153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 -- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd784159180 msgr2=0x7fd7841595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd784159180 0x7fd7841595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 -- 172.21.15.144:0/3973335224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78415cd80 con 0x7fd784151b50 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7827fc700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 0x7fd7841521f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd774011cd0 tx=0x7fd774009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400fa00 con 0x7fd784151b50 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77400fb60 con 0x7fd784151b50 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7840d15e0 con 0x7fd784151b50 2023-11-28T11:52:48.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774004030 con 0x7fd784151b50 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.971+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78415cf10 con 0x7fd784151b50 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.972+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd784153930 con 0x7fd784151b50 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.973+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd774024070 con 0x7fd784151b50 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.973+0000 7fd76b7fe700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd7700682f0 0x7fd77006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.973+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fd774085ef0 con 0x7fd784151b50 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.973+0000 7fd782ffd700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd7700682f0 0x7fd77006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.973+0000 7fd782ffd700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd7700682f0 0x7fd77006a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd778009de0 tx=0x7fd778009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:48.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:48.977+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd774052930 con 0x7fd784151b50 2023-11-28T11:52:49.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.161+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7840745c0 con 0x7fd784151b50 2023-11-28T11:52:49.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.813+0000 7fd76b7fe700 1 -- 172.21.15.144:0/3973335224 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd774022020 con 0x7fd784151b50 2023-11-28T11:52:49.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.818+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd7700682f0 msgr2=0x7fd77006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.818+0000 7fd7897d6700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd7700682f0 0x7fd77006a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd778009de0 tx=0x7fd778009210 comp rx=0 tx=0).stop 2023-11-28T11:52:49.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 msgr2=0x7fd7841521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 0x7fd7841521f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd774011cd0 tx=0x7fd774009ed0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 shutdown_connections 2023-11-28T11:52:49.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd7700682f0 0x7fd77006a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd784159180 0x7fd7841595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd7840c5f40 0x7fd784153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 --2- 172.21.15.144:0/3973335224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd784151b50 0x7fd7841521f0 secure :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd774011cd0 tx=0x7fd774009ed0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.819+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 >> 172.21.15.144:0/3973335224 conn(0x7fd784072600 msgr2=0x7fd78406a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:49.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.821+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 shutdown_connections 2023-11-28T11:52:49.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.821+0000 7fd7897d6700 1 -- 172.21.15.144:0/3973335224 wait complete. 2023-11-28T11:52:49.830 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' created 2023-11-28T11:52:49.843 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:52:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.967+0000 7f02a8523700 1 Processor -- start 2023-11-28T11:52:49.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.967+0000 7f02a8523700 1 -- start start 2023-11-28T11:52:49.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a8523700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 0x7f02a00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a8523700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a00cc5f0 con 0x7f02a006ba90 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a8523700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a015d470 con 0x7f02a00cbdf0 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a8523700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a015d750 con 0x7f02a015cef0 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a6ac0700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f02a006ba90 0x7f02a00ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53970/0 (socket says 172.21.15.144:53970) 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a6ac0700 1 -- 172.21.15.144:0/2226391406 learned_addr learned my addr 172.21.15.144:0/2226391406 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a62bf700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 0x7f02a00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481374058 0 0) 0x7f02a00cc5f0 con 0x7f02a006ba90 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f028c0035b0 con 0x7f02a006ba90 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658307955 0 0) 0x7f02a015d470 con 0x7f02a00cbdf0 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a00cc5f0 con 0x7f02a00cbdf0 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243944138 0 0) 0x7f028c0035b0 con 0x7f02a006ba90 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 msgr2=0x7f02a00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:49.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 0x7f02a00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 >> v1:172.21.15.177:6789/0 conn(0x7f02a00cbdf0 legacy=0x7f02a006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a015da30 con 0x7f02a006ba90 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f02980019f0 con 0x7f02a006ba90 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.968+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 >> v1:172.21.15.144:6789/0 conn(0x7f02a006ba90 legacy=0x7f02a00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 0x7f028c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f028c004120 0x7f028c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f028c004d90 0x7f028c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a015d750 con 0x7f028c004d90 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c00bac0 con 0x7f028c004120 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c00bc80 con 0x7f02a015cef0 2023-11-28T11:52:49.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f028c004d90 0x7f028c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a62bf700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 0x7f028c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a5abe700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f028c004120 0x7f028c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 -- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 msgr2=0x7f028c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02a015cef0 0x7f028c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 -- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f028c004120 msgr2=0x7f028c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f028c004120 0x7f028c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 -- 172.21.15.144:0/2226391406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c0035b0 con 0x7f028c004d90 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a6ac0700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f028c004d90 0x7f028c005200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0298002230 tx=0x7f028c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aee4dc23aae6598 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980072d0 con 0x7f028c004d90 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298007430 con 0x7f028c004d90 2023-11-28T11:52:49.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.969+0000 7f02a52bd700 1 -- 172.21.15.144:0/2226391406 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980077a0 con 0x7f028c004d90 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a8523700 1 -- 172.21.15.144:0/2226391406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f028c004d90 msgr2=0x7f028c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a8523700 1 --2- 172.21.15.144:0/2226391406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f028c004d90 0x7f028c005200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0298002230 tx=0x7f028c005880 comp rx=0 tx=0).stop 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a6ac0700 1 -- 172.21.15.144:0/2226391406 reap_dead start 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a8523700 1 -- 172.21.15.144:0/2226391406 shutdown_connections 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a8523700 1 -- 172.21.15.144:0/2226391406 >> 172.21.15.144:0/2226391406 conn(0x7f02a00d0000 msgr2=0x7f02a00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a8523700 1 -- 172.21.15.144:0/2226391406 shutdown_connections 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.970+0000 7f02a8523700 1 -- 172.21.15.144:0/2226391406 wait complete. 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.971+0000 7f02a8523700 1 Processor -- start 2023-11-28T11:52:49.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.971+0000 7f02a8523700 1 -- start start 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a8523700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f028c004d90 0x7f02a015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a8523700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 0x7f02a00c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a8523700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a009c3c0 0x7f02a009c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a8523700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c68d0 con 0x7f02a00c5570 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a8523700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a009cda0 con 0x7f02a009c3c0 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a8523700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c6030 con 0x7f028c004d90 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 0x7f02a00c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 0x7f02a00c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37268/0 (socket says 172.21.15.144:37268) 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 -- 172.21.15.144:0/2853583144 learned_addr learned my addr 172.21.15.144:0/2853583144 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:49.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a62bf700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f028c004d90 0x7f02a015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a6ac0700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a009c3c0 0x7f02a009c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 -- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f028c004d90 msgr2=0x7f02a015b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f028c004d90 0x7f02a015b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 -- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a009c3c0 msgr2=0x7f02a009c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a009c3c0 0x7f02a009c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 -- 172.21.15.144:0/2853583144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a015dff0 con 0x7f02a00c5570 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f02a5abe700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 0x7f02a00c59e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f029000f7e0 tx=0x7f029000faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.972+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290001520 con 0x7f02a00c5570 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.973+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02900052e0 con 0x7f02a00c5570 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.973+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290003680 con 0x7f02a00c5570 2023-11-28T11:52:49.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.973+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a015dd10 con 0x7f02a00c5570 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.973+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a00af400 con 0x7f02a00c5570 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.973+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a00c38b0 con 0x7f02a00c5570 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.974+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0290003800 con 0x7f02a00c5570 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.974+0000 7f0296ffd700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02840682f0 0x7f028406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.974+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0290085cc0 con 0x7f02a00c5570 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.974+0000 7f02a62bf700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02840682f0 0x7f028406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.975+0000 7f02a62bf700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02840682f0 0x7f028406a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f02a009dac0 tx=0x7f029c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:49.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:49.979+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f029008c080 con 0x7f02a00c5570 2023-11-28T11:52:50.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.205+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f02a00d0e60 con 0x7f02a00c5570 2023-11-28T11:52:50.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.213+0000 7f0296ffd700 1 -- 172.21.15.144:0/2853583144 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0290052490 con 0x7f02a00c5570 2023-11-28T11:52:50.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.221+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02840682f0 msgr2=0x7f028406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.221+0000 7f02a8523700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02840682f0 0x7f028406a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f02a009dac0 tx=0x7f029c009210 comp rx=0 tx=0).stop 2023-11-28T11:52:50.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.221+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 msgr2=0x7f02a00c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.221+0000 7f02a8523700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 0x7f02a00c59e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f029000f7e0 tx=0x7f029000faf0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.224 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.222+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 shutdown_connections 2023-11-28T11:52:50.224 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.222+0000 7f02a8523700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02840682f0 0x7f028406a7a0 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f02a009dac0 tx=0x7f029c009210 comp rx=0 tx=0).stop 2023-11-28T11:52:50.225 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.222+0000 7f02a8523700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02a009c3c0 0x7f02a009c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.225 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.222+0000 7f02a8523700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f028c004d90 0x7f02a015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.225 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.222+0000 7f02a8523700 1 --2- 172.21.15.144:0/2853583144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02a00c5570 0x7f02a00c59e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.222+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 >> 172.21.15.144:0/2853583144 conn(0x7f02a00d0000 msgr2=0x7f02a00af990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:50.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.226+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 shutdown_connections 2023-11-28T11:52:50.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.227+0000 7f02a8523700 1 -- 172.21.15.144:0/2853583144 wait complete. 2023-11-28T11:52:50.230 INFO:teuthology.orchestra.run.smithi144.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T11:52:50.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.364+0000 7f898a04c700 1 Processor -- start 2023-11-28T11:52:50.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f898a04c700 1 -- start start 2023-11-28T11:52:50.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f898a04c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 0x7f8984167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f898a04c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898415d060 con 0x7f898406dee0 2023-11-28T11:52:50.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f898a04c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898415d340 con 0x7f8984071040 2023-11-28T11:52:50.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f898a04c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8984161550 con 0x7f89841658c0 2023-11-28T11:52:50.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f8982ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f898406dee0 0x7f898406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:53988/0 (socket says 172.21.15.144:53988) 2023-11-28T11:52:50.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.365+0000 7f8982ffd700 1 -- 172.21.15.144:0/1767821935 learned_addr learned my addr 172.21.15.144:0/1767821935 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:50.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 0x7f8984167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934175909 0 0) 0x7f898415d060 con 0x7f898406dee0 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89700035b0 con 0x7f898406dee0 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277930437 0 0) 0x7f89700035b0 con 0x7f898406dee0 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 msgr2=0x7f8984167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 0x7f8984167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 >> v1:172.21.15.177:6789/0 conn(0x7f8984071040 legacy=0x7f8984071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898415d620 con 0x7f898406dee0 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f896c001670 con 0x7f898406dee0 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 >> v1:172.21.15.144:6789/0 conn(0x7f898406dee0 legacy=0x7f898406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970003a30 0x7f8970003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 0x7f8970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8970004d90 0x7f8970005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8984161550 con 0x7f8970004d90 2023-11-28T11:52:50.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897000bbd0 con 0x7f8970003a30 2023-11-28T11:52:50.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897000bd90 con 0x7f89841658c0 2023-11-28T11:52:50.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8970004d90 0x7f8970005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f89837fe700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970003a30 0x7f8970003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8982ffd700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 0x7f8970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 -- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 msgr2=0x7f8970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:50.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89841658c0 0x7f8970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 -- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970003a30 msgr2=0x7f8970003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970003a30 0x7f8970003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.366+0000 7f8983fff700 1 -- 172.21.15.144:0/1767821935 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89700035b0 con 0x7f8970004d90 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f8983fff700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8970004d90 0x7f8970005200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f89840c3a30 tx=0x7f8970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d29575c95b9e94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c00eba0 con 0x7f8970004d90 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c00f040 con 0x7f8970004d90 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f89827fc700 1 -- 172.21.15.144:0/1767821935 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c014480 con 0x7f8970004d90 2023-11-28T11:52:50.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f898a04c700 1 -- 172.21.15.144:0/1767821935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8970004d90 msgr2=0x7f8970005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f898a04c700 1 --2- 172.21.15.144:0/1767821935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8970004d90 0x7f8970005200 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f89840c3a30 tx=0x7f8970005880 comp rx=0 tx=0).stop 2023-11-28T11:52:50.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f8983fff700 1 -- 172.21.15.144:0/1767821935 reap_dead start 2023-11-28T11:52:50.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f898a04c700 1 -- 172.21.15.144:0/1767821935 shutdown_connections 2023-11-28T11:52:50.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f898a04c700 1 -- 172.21.15.144:0/1767821935 >> 172.21.15.144:0/1767821935 conn(0x7f89840d0000 msgr2=0x7f89840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:50.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f898a04c700 1 -- 172.21.15.144:0/1767821935 shutdown_connections 2023-11-28T11:52:50.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.367+0000 7f898a04c700 1 -- 172.21.15.144:0/1767821935 wait complete. 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 Processor -- start 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 -- start start 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89840c5ce0 0x7f8984158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89841563d0 0x7f89840c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 0x7f898415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898415fb50 con 0x7f898415f170 2023-11-28T11:52:50.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.369+0000 7f898a04c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898414ad80 con 0x7f89841563d0 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f898a04c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89840c60b0 con 0x7f89840c5ce0 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 0x7f898415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 0x7f898415f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37290/0 (socket says 172.21.15.144:37290) 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 -- 172.21.15.144:0/2674669741 learned_addr learned my addr 172.21.15.144:0/2674669741 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8982ffd700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89841563d0 0x7f89840c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f89837fe700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89840c5ce0 0x7f8984158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 -- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89840c5ce0 msgr2=0x7f8984158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89840c5ce0 0x7f8984158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 -- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89841563d0 msgr2=0x7f89840c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89841563d0 0x7f89840c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 -- 172.21.15.144:0/2674669741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898415ccd0 con 0x7f898415f170 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f89837fe700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89840c5ce0 0x7f8984158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8982ffd700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89841563d0 0x7f89840c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f8983fff700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 0x7f898415f610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f897c017040 tx=0x7f897c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c007000 con 0x7f898415f170 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c014d40 con 0x7f898415f170 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898415d900 con 0x7f898415f170 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c003c60 con 0x7f898415f170 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.370+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898406a270 con 0x7f898415f170 2023-11-28T11:52:50.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.371+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89840720a0 con 0x7f898415f170 2023-11-28T11:52:50.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.372+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f897c01d420 con 0x7f898415f170 2023-11-28T11:52:50.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.372+0000 7f897bfff700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89640682f0 0x7f896406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.372+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f897c087470 con 0x7f898415f170 2023-11-28T11:52:50.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.372+0000 7f89837fe700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89640682f0 0x7f896406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.372+0000 7f89837fe700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89640682f0 0x7f896406a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8974007fd0 tx=0x7f8974009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:50.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.376+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f897c053d30 con 0x7f898415f170 2023-11-28T11:52:50.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.560+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89841595d0 con 0x7f898415f170 2023-11-28T11:52:50.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.561+0000 7f897bfff700 1 -- 172.21.15.144:0/2674669741 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+17982 (secure 0 0 0) 0x7f897c057350 con 0x7f898415f170 2023-11-28T11:52:50.563 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:50.564 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":45,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:50.207127+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89640682f0 msgr2=0x7f896406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89640682f0 0x7f896406a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8974007fd0 tx=0x7f8974009040 comp rx=0 tx=0).stop 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 msgr2=0x7f898415f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 0x7f898415f610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f897c017040 tx=0x7f897c009750 comp rx=0 tx=0).stop 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 shutdown_connections 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f898415f170 0x7f898415f610 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89640682f0 0x7f896406a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.566 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89840c5ce0 0x7f8984158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.567 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 --2- 172.21.15.144:0/2674669741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89841563d0 0x7f89840c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.567 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 >> 172.21.15.144:0/2674669741 conn(0x7f89840d0000 msgr2=0x7f89840cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:50.567 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 shutdown_connections 2023-11-28T11:52:50.567 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.564+0000 7f898a04c700 1 -- 172.21.15.144:0/2674669741 wait complete. 2023-11-28T11:52:50.578 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:50.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.696+0000 7fcbf765b700 1 Processor -- start 2023-11-28T11:52:50.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.696+0000 7fcbf765b700 1 -- start start 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.697+0000 7fcbf765b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcbf00d1330 0x7fcbf00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.697+0000 7fcbf765b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00b0230 con 0x7fcbf00d01b0 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.697+0000 7fcbf765b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00b4570 con 0x7fcbf00b20b0 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.697+0000 7fcbf765b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf015cec0 con 0x7fcbf00d1330 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.697+0000 7fcbf53f7700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcbf00d01b0 0x7fcbf00d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54006/0 (socket says 172.21.15.144:54006) 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.697+0000 7fcbf53f7700 1 -- 172.21.15.144:0/3057612319 learned_addr learned my addr 172.21.15.144:0/3057612319 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:50.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbf4bf6700 1 --2- 172.21.15.144:0/3057612319 >> v2:172.21.15.177:3300/0 conn(0x7fcbf00d1330 0x7fcbf00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbe3fff700 1 -- 172.21.15.144:0/3057612319 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947397482 0 0) 0x7fcbf00b0230 con 0x7fcbf00d01b0 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbe3fff700 1 -- 172.21.15.144:0/3057612319 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbdc0035b0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbe3fff700 1 -- 172.21.15.144:0/3057612319 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329846479 0 0) 0x7fcbdc0035b0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/3057612319 >> v1:172.21.15.177:6789/0 conn(0x7fcbf00b20b0 legacy=0x7fcbf00b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/3057612319 >> v1:172.21.15.144:6789/0 conn(0x7fcbf00d01b0 legacy=0x7fcbf00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/3057612319 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00b4850 con 0x7fcbf00d1330 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbf4bf6700 1 --2- 172.21.15.144:0/3057612319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf00b06c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009840 tx=0x7fcbec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ff9927e54622e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbe3fff700 1 -- 172.21.15.144:0/3057612319 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4001670 con 0x7fcbf00d1330 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbe3fff700 1 -- 172.21.15.144:0/3057612319 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec004030 con 0x7fcbf00d1330 2023-11-28T11:52:50.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.698+0000 7fcbe3fff700 1 -- 172.21.15.144:0/3057612319 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0043a0 con 0x7fcbf00d1330 2023-11-28T11:52:50.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 -- 172.21.15.144:0/3057612319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 msgr2=0x7fcbf00b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 --2- 172.21.15.144:0/3057612319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf00b06c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009840 tx=0x7fcbec009b50 comp rx=0 tx=0).stop 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 -- 172.21.15.144:0/3057612319 shutdown_connections 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 --2- 172.21.15.144:0/3057612319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf00b06c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 -- 172.21.15.144:0/3057612319 >> 172.21.15.144:0/3057612319 conn(0x7fcbf009c3f0 msgr2=0x7fcbf00d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 -- 172.21.15.144:0/3057612319 shutdown_connections 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.699+0000 7fcbf765b700 1 -- 172.21.15.144:0/3057612319 wait complete. 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.700+0000 7fcbf765b700 1 Processor -- start 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.700+0000 7fcbf765b700 1 -- start start 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf765b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf00b20b0 0x7fcbf01f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf765b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 0x7fcbf015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf765b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf015f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf765b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cc0f0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf765b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cb830 con 0x7fcbf00d1330 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf765b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cbaa0 con 0x7fcbf00b20b0 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 0x7fcbf015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 0x7fcbf015f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37302/0 (socket says 172.21.15.144:37302) 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/335566789 learned_addr learned my addr 172.21.15.144:0/335566789 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf5bf8700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf53f7700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf00b20b0 0x7fcbf01f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf00b20b0 msgr2=0x7fcbf01f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf00b20b0 0x7fcbf01f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 msgr2=0x7fcbf015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf015f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 -- 172.21.15.144:0/335566789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec0094a0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.701+0000 7fcbf4bf6700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 0x7fcbf015f2c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009b90 tx=0x7fcbec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.702+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec003e80 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.702+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec010590 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.702+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf00afec0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.702+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec010920 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.702+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf00afbe0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.703+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf00b2600 con 0x7fcbf00d01b0 2023-11-28T11:52:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.703+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbec00f050 con 0x7fcbf00d01b0 2023-11-28T11:52:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.703+0000 7fcbe1ffb700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcbd40682f0 0x7fcbd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.704+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbec087460 con 0x7fcbf00d01b0 2023-11-28T11:52:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.704+0000 7fcbf53f7700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcbd40682f0 0x7fcbd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.704+0000 7fcbf53f7700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcbd40682f0 0x7fcbd406a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4005a70 tx=0x7fcbe4001c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:50.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.708+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbec053bd0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.893+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbf0068690 con 0x7fcbf00d01b0 2023-11-28T11:52:50.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.894+0000 7fcbe1ffb700 1 -- 172.21.15.144:0/335566789 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+17982 (secure 0 0 0) 0x7fcbec0571f0 con 0x7fcbf00d01b0 2023-11-28T11:52:50.896 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:50.896 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":45,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:50.207127+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:50.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcbd40682f0 msgr2=0x7fcbd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcbd40682f0 0x7fcbd406a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4005a70 tx=0x7fcbe4001c00 comp rx=0 tx=0).stop 2023-11-28T11:52:50.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 msgr2=0x7fcbf015f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 0x7fcbf015f2c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009b90 tx=0x7fcbec004750 comp rx=0 tx=0).stop 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 shutdown_connections 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcbd40682f0 0x7fcbd406a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcbf00d1330 0x7fcbf015f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcbf00b20b0 0x7fcbf01f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 --2- 172.21.15.144:0/335566789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf00d01b0 0x7fcbf015f2c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 >> 172.21.15.144:0/335566789 conn(0x7fcbf009c3f0 msgr2=0x7fcbf006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 shutdown_connections 2023-11-28T11:52:50.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:50.896+0000 7fcbf765b700 1 -- 172.21.15.144:0/335566789 wait complete. 2023-11-28T11:52:50.911 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:52:51.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.032+0000 7f0580354700 1 Processor -- start 2023-11-28T11:52:51.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.032+0000 7f0580354700 1 -- start start 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.033+0000 7f0580354700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0578154e70 0x7f05780cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.033+0000 7f0580354700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0578071750 0x7f057815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.033+0000 7f0580354700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0578155240 con 0x7f05780cf360 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.033+0000 7f0580354700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057815b100 con 0x7f0578154e70 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.033+0000 7f0580354700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057815b3e0 con 0x7f0578071750 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0578154e70 0x7f05780cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057d8ef700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f05780cf360 0x7f0578071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54022/0 (socket says 172.21.15.144:54022) 2023-11-28T11:52:51.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0578154e70 0x7f05780cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37308/0 (socket says 172.21.15.144:37308) 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057d8ef700 1 -- 172.21.15.144:0/381018245 learned_addr learned my addr 172.21.15.144:0/381018245 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e8f1700 1 --2- 172.21.15.144:0/381018245 >> v2:172.21.15.177:3300/0 conn(0x7f0578071750 0x7f057815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 -- 172.21.15.144:0/381018245 >> v2:172.21.15.177:3300/0 conn(0x7f0578071750 msgr2=0x7f057815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 --2- 172.21.15.144:0/381018245 >> v2:172.21.15.177:3300/0 conn(0x7f0578071750 0x7f057815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 -- 172.21.15.144:0/381018245 >> v1:172.21.15.144:6789/0 conn(0x7f05780cf360 legacy=0x7f0578071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 -- 172.21.15.144:0/381018245 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057815b6c0 con 0x7f0578154e70 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057e0f0700 1 --2- 172.21.15.144:0/381018245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0578154e70 0x7f05780cdbd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f05780cb0a0 tx=0x7f056c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b035e14701dd3576 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057d0ee700 1 -- 172.21.15.144:0/381018245 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c011830 con 0x7f0578154e70 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057d0ee700 1 -- 172.21.15.144:0/381018245 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c011990 con 0x7f0578154e70 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.034+0000 7f057d0ee700 1 -- 172.21.15.144:0/381018245 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c011d20 con 0x7f0578154e70 2023-11-28T11:52:51.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 -- 172.21.15.144:0/381018245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0578154e70 msgr2=0x7f05780cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 --2- 172.21.15.144:0/381018245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0578154e70 0x7f05780cdbd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f05780cb0a0 tx=0x7f056c009b20 comp rx=0 tx=0).stop 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 -- 172.21.15.144:0/381018245 shutdown_connections 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 --2- 172.21.15.144:0/381018245 >> v2:172.21.15.177:3300/0 conn(0x7f0578071750 0x7f057815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 --2- 172.21.15.144:0/381018245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0578154e70 0x7f05780cdbd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 -- 172.21.15.144:0/381018245 >> 172.21.15.144:0/381018245 conn(0x7f05780d0000 msgr2=0x7f05780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 -- 172.21.15.144:0/381018245 shutdown_connections 2023-11-28T11:52:51.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.035+0000 7f0580354700 1 -- 172.21.15.144:0/381018245 wait complete. 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.036+0000 7f0580354700 1 Processor -- start 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.036+0000 7f0580354700 1 -- start start 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0578071750 0x7f0578154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 0x7f057806e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0578154e70 0x7f05780cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05780b4920 con 0x7f05780cf360 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05780b4ba0 con 0x7f0578071750 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05780b4e80 con 0x7f0578154e70 2023-11-28T11:52:51.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 0x7f057806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 0x7f057806e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37316/0 (socket says 172.21.15.144:37316) 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 -- 172.21.15.144:0/2045568205 learned_addr learned my addr 172.21.15.144:0/2045568205 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057e8f1700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0578154e70 0x7f05780cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057e0f0700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0578071750 0x7f0578154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 -- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0578154e70 msgr2=0x7f05780cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0578154e70 0x7f05780cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 -- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0578071750 msgr2=0x7f0578154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0578071750 0x7f0578154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 -- 172.21.15.144:0/2045568205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0094a0 con 0x7f05780cf360 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057e8f1700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0578154e70 0x7f05780cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057e0f0700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0578071750 0x7f0578154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:51.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f057d8ef700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 0x7f057806e550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f057400c520 tx=0x7f057400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574005d70 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05740070d0 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574007440 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.037+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05780af520 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.038+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057815b9a0 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.038+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057815adb0 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.039+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0574019030 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.039+0000 7f056affd700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f055c0682f0 0x7f055c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.039+0000 7f057e0f0700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f055c0682f0 0x7f055c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.039+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0574084ff0 con 0x7f05780cf360 2023-11-28T11:52:51.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.040+0000 7f057e0f0700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f055c0682f0 0x7f055c06a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f056c0090b0 tx=0x7f056c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:51.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.044+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f057404b5e0 con 0x7f05780cf360 2023-11-28T11:52:51.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:51.273+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05780cf140 con 0x7f05780cf360 2023-11-28T11:52:52.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.224+0000 7f056affd700 1 -- 172.21.15.144:0/2045568205 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f05740170b0 con 0x7f05780cf360 2023-11-28T11:52:52.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f055c0682f0 msgr2=0x7f055c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f055c0682f0 0x7f055c06a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f056c0090b0 tx=0x7f056c01c040 comp rx=0 tx=0).stop 2023-11-28T11:52:52.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 msgr2=0x7f057806e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 0x7f057806e550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f057400c520 tx=0x7f057400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.229 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 shutdown_connections 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f055c0682f0 0x7f055c06a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0578154e70 0x7f05780cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0578071750 0x7f0578154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 --2- 172.21.15.144:0/2045568205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05780cf360 0x7f057806e550 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 >> 172.21.15.144:0/2045568205 conn(0x7f05780d0000 msgr2=0x7f05780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 shutdown_connections 2023-11-28T11:52:52.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.227+0000 7f0580354700 1 -- 172.21.15.144:0/2045568205 wait complete. 2023-11-28T11:52:52.244 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:52:52.244 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:52:52.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.372+0000 7f9e22566700 1 Processor -- start 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.372+0000 7f9e22566700 1 -- start start 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.372+0000 7f9e22566700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9e1c0cbdf0 0x7f9e1c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e22566700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1c0cc5f0 con 0x7f9e1c06ba90 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e22566700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1c15d470 con 0x7f9e1c15cef0 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e22566700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1c15d750 con 0x7f9e1c0cbdf0 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9e1c06ba90 0x7f9e1c0ca650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54034/0 (socket says 172.21.15.144:54034) 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1b7fe700 1 -- 172.21.15.144:0/412644296 learned_addr learned my addr 172.21.15.144:0/412644296 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e20b03700 1 --2- 172.21.15.144:0/412644296 >> v2:172.21.15.177:3300/0 conn(0x7f9e1c0cbdf0 0x7f9e1c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175866263 0 0) 0x7f9e1c0cc5f0 con 0x7f9e1c06ba90 2023-11-28T11:52:52.387 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e040035b0 con 0x7f9e1c06ba90 2023-11-28T11:52:52.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3855627056 0 0) 0x7f9e040035b0 con 0x7f9e1c06ba90 2023-11-28T11:52:52.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 >> v2:172.21.15.177:3300/0 conn(0x7f9e1c0cbdf0 msgr2=0x7f9e1c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:52.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 --2- 172.21.15.144:0/412644296 >> v2:172.21.15.177:3300/0 conn(0x7f9e1c0cbdf0 0x7f9e1c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 >> v1:172.21.15.177:6789/0 conn(0x7f9e1c15cef0 legacy=0x7f9e1c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c15da30 con 0x7f9e1c06ba90 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f9e10001670 con 0x7f9e1c06ba90 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 >> v1:172.21.15.144:6789/0 conn(0x7f9e1c06ba90 legacy=0x7f9e1c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e04003a00 0x7f9e04003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e04004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e04005200 0x7f9e0400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c15d750 con 0x7f9e04004850 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0400c080 con 0x7f9e04005200 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0400c240 con 0x7f9e04003a00 2023-11-28T11:52:52.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e04004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e20b03700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e04005200 0x7f9e0400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.373+0000 7f9e1bfff700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e04003a00 0x7f9e04003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1b7fe700 1 -- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e04003a00 msgr2=0x7f9e04003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e04003a00 0x7f9e04003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1b7fe700 1 -- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e04005200 msgr2=0x7f9e0400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e04005200 0x7f9e0400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e20b03700 1 -- 172.21.15.144:0/412644296 reap_dead start 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1b7fe700 1 -- 172.21.15.144:0/412644296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040035b0 con 0x7f9e04004850 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e04004cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c240 tx=0x7f9e04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e448c76f3c7fe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10007c10 con 0x7f9e04004850 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000cd60 con 0x7f9e04004850 2023-11-28T11:52:52.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e1affd700 1 -- 172.21.15.144:0/412644296 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10005ba0 con 0x7f9e04004850 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e22566700 1 -- 172.21.15.144:0/412644296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 msgr2=0x7f9e04004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e22566700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e04004cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c240 tx=0x7f9e04005880 comp rx=0 tx=0).stop 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e22566700 1 -- 172.21.15.144:0/412644296 shutdown_connections 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e22566700 1 --2- 172.21.15.144:0/412644296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e04004cc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.374+0000 7f9e22566700 1 -- 172.21.15.144:0/412644296 >> 172.21.15.144:0/412644296 conn(0x7f9e1c0d0000 msgr2=0x7f9e1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.375+0000 7f9e22566700 1 -- 172.21.15.144:0/412644296 shutdown_connections 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.375+0000 7f9e22566700 1 -- 172.21.15.144:0/412644296 wait complete. 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.376+0000 7f9e22566700 1 Processor -- start 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.376+0000 7f9e22566700 1 -- start start 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e1c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e1c09a5a0 0x7f9e1c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.391 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1c09a9f0 0x7f9e1c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0c41d0 con 0x7f9e04004850 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0c6240 con 0x7f9e1c09a5a0 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0c6520 con 0x7f9e1c09a9f0 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e1c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e1c157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37334/0 (socket says 172.21.15.144:37334) 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 -- 172.21.15.144:0/3072135907 learned_addr learned my addr 172.21.15.144:0/3072135907 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e1c09a5a0 0x7f9e1c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e20b03700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1c09a9f0 0x7f9e1c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 -- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1c09a9f0 msgr2=0x7f9e1c0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1c09a9f0 0x7f9e1c0b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 -- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e1c09a5a0 msgr2=0x7f9e1c0c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:52.392 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e1c09a5a0 0x7f9e1c0c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 -- 172.21.15.144:0/3072135907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c15dff0 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e1c09a5a0 0x7f9e1c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e1bfff700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e1c157730 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00f820 tx=0x7f9e0c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0052e0 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c005460 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1c15dd10 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0038c0 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.377+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1c0af400 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.378+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1c0d0e60 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.379+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e0c003a20 con 0x7f9e04004850 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.379+0000 7f9e18ff9700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e080682f0 0x7f9e0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:52.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.379+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e0c085bc0 con 0x7f9e04004850 2023-11-28T11:52:52.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.379+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e080682f0 0x7f9e0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:52.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.380+0000 7f9e1b7fe700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e080682f0 0x7f9e0806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e10015040 tx=0x7f9e10007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:52.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.383+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e0c052420 con 0x7f9e04004850 2023-11-28T11:52:52.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:52.608+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9e1c0ceb00 con 0x7f9e04004850 2023-11-28T11:52:52.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:52:52.801 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:52:53.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.229+0000 7f9e18ff9700 1 -- 172.21.15.144:0/3072135907 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e0c003cb0 con 0x7f9e04004850 2023-11-28T11:52:53.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e080682f0 msgr2=0x7f9e0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e080682f0 0x7f9e0806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e10015040 tx=0x7f9e10007000 comp rx=0 tx=0).stop 2023-11-28T11:52:53.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 msgr2=0x7f9e1c157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e1c157730 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00f820 tx=0x7f9e0c00fbe0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 shutdown_connections 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04004850 0x7f9e1c157730 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e080682f0 0x7f9e0806a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e1c09a9f0 0x7f9e1c0b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 --2- 172.21.15.144:0/3072135907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e1c09a5a0 0x7f9e1c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.231+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 >> 172.21.15.144:0/3072135907 conn(0x7f9e1c0d0000 msgr2=0x7f9e1c0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.232+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 shutdown_connections 2023-11-28T11:52:53.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.232+0000 7f9e22566700 1 -- 172.21.15.144:0/3072135907 wait complete. 2023-11-28T11:52:53.246 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:52:53.247 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:53.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.366+0000 7fee4e181700 1 Processor -- start 2023-11-28T11:52:53.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.366+0000 7fee4e181700 1 -- start start 2023-11-28T11:52:53.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee4e181700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 0x7fee480b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee4e181700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4813ff00 con 0x7fee480b2e70 2023-11-28T11:52:53.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee4e181700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee481404c0 con 0x7fee480cd8c0 2023-11-28T11:52:53.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee4e181700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee48140dc0 con 0x7fee48140950 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee47fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fee480b2e70 0x7fee4806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54058/0 (socket says 172.21.15.144:54058) 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee47fff700 1 -- 172.21.15.144:0/2057037636 learned_addr learned my addr 172.21.15.144:0/2057037636 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee46ffd700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 0x7fee480b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185381215 0 0) 0x7fee4813ff00 con 0x7fee480b2e70 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee340035b0 con 0x7fee480b2e70 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1187753281 0 0) 0x7fee340035b0 con 0x7fee480b2e70 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 msgr2=0x7fee480b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 0x7fee480b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 >> v1:172.21.15.177:6789/0 conn(0x7fee480cd8c0 legacy=0x7fee480cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4815aa80 con 0x7fee480b2e70 2023-11-28T11:52:53.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.367+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fee400019f0 con 0x7fee480b2e70 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 >> v1:172.21.15.144:6789/0 conn(0x7fee480b2e70 legacy=0x7fee4806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 0x7fee34003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee34003fb0 0x7fee34004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee34004d90 0x7fee34005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48140dc0 con 0x7fee34003fb0 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3400bba0 con 0x7fee34004d90 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3400bd60 con 0x7fee48140950 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee34003fb0 0x7fee34004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee477fe700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 0x7fee34003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee47fff700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee34004d90 0x7fee34005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 -- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 msgr2=0x7fee34003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee48140950 0x7fee34003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 -- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee34004d90 msgr2=0x7fee34005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee34004d90 0x7fee34005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 -- 172.21.15.144:0/2057037636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340035b0 con 0x7fee34003fb0 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee47fff700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee34004d90 0x7fee34005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee46ffd700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee34003fb0 0x7fee34004850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fee480cea30 tx=0x7fee34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf48b4c19f86a98b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800e6c0 con 0x7fee34003fb0 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3800e820 con 0x7fee34003fb0 2023-11-28T11:52:53.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.368+0000 7fee467fc700 1 -- 172.21.15.144:0/2057037636 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800eb90 con 0x7fee34003fb0 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee4e181700 1 -- 172.21.15.144:0/2057037636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee34003fb0 msgr2=0x7fee34004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee4e181700 1 --2- 172.21.15.144:0/2057037636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee34003fb0 0x7fee34004850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fee480cea30 tx=0x7fee34005880 comp rx=0 tx=0).stop 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee47fff700 1 -- 172.21.15.144:0/2057037636 reap_dead start 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee4e181700 1 -- 172.21.15.144:0/2057037636 shutdown_connections 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee4e181700 1 -- 172.21.15.144:0/2057037636 >> 172.21.15.144:0/2057037636 conn(0x7fee4806aed0 msgr2=0x7fee4806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee4e181700 1 -- 172.21.15.144:0/2057037636 shutdown_connections 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.369+0000 7fee4e181700 1 -- 172.21.15.144:0/2057037636 wait complete. 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.370+0000 7fee4e181700 1 Processor -- start 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.370+0000 7fee4e181700 1 -- start start 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee4e181700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee34003fb0 0x7fee4806ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee4e181700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 0x7fee4813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee4e181700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee48156ae0 0x7fee4813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee4e181700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c55f0 con 0x7fee48156670 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee4e181700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c49e0 con 0x7fee48156ae0 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee4e181700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c4cc0 con 0x7fee34003fb0 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 0x7fee4813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 0x7fee4813ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37356/0 (socket says 172.21.15.144:37356) 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 -- 172.21.15.144:0/345822742 learned_addr learned my addr 172.21.15.144:0/345822742 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee477fe700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee34003fb0 0x7fee4806ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee47fff700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee48156ae0 0x7fee4813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 -- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee34003fb0 msgr2=0x7fee4806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee34003fb0 0x7fee4806ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 -- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee48156ae0 msgr2=0x7fee4813d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee48156ae0 0x7fee4813d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee46ffd700 1 -- 172.21.15.144:0/345822742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480b35f0 con 0x7fee48156670 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee477fe700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee34003fb0 0x7fee4806ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.371+0000 7fee47fff700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee48156ae0 0x7fee4813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.372+0000 7fee46ffd700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 0x7fee4813ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fee38000c00 tx=0x7fee380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.372+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800e6c0 con 0x7fee48156670 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.372+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380038b0 con 0x7fee48156670 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.372+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee480b3310 con 0x7fee48156670 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.372+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38006e60 con 0x7fee48156670 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.372+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee48140150 con 0x7fee48156670 2023-11-28T11:52:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.373+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee48068690 con 0x7fee48156670 2023-11-28T11:52:53.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.373+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fee38025070 con 0x7fee48156670 2023-11-28T11:52:53.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.374+0000 7fee3ffff700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fee28068320 0x7fee2806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.374+0000 7fee477fe700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fee28068320 0x7fee2806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.374+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee38087560 con 0x7fee48156670 2023-11-28T11:52:53.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.374+0000 7fee477fe700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fee28068320 0x7fee2806a7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee30000c00 tx=0x7fee30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:53.382 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.378+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fee38053dc0 con 0x7fee48156670 2023-11-28T11:52:53.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.606+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee480b2aa0 con 0x7fee48156670 2023-11-28T11:52:53.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.606+0000 7fee3ffff700 1 -- 172.21.15.144:0/345822742 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fee380573e0 con 0x7fee48156670 2023-11-28T11:52:53.609 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:53.609 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fee28068320 msgr2=0x7fee2806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fee28068320 0x7fee2806a7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee30000c00 tx=0x7fee30009040 comp rx=0 tx=0).stop 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 msgr2=0x7fee4813ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 0x7fee4813ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fee38000c00 tx=0x7fee380047c0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 shutdown_connections 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fee28068320 0x7fee2806a7d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee48156ae0 0x7fee4813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fee34003fb0 0x7fee4806ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 --2- 172.21.15.144:0/345822742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48156670 0x7fee4813ccf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.608+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 >> 172.21.15.144:0/345822742 conn(0x7fee4806aed0 msgr2=0x7fee480cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:53.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.609+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 shutdown_connections 2023-11-28T11:52:53.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.609+0000 7fee4e181700 1 -- 172.21.15.144:0/345822742 wait complete. 2023-11-28T11:52:53.612 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:52:53.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:52:53.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.742+0000 7f38eb094700 1 Processor -- start 2023-11-28T11:52:53.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.742+0000 7f38eb094700 1 -- start start 2023-11-28T11:52:53.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38eb094700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406e9f0 0x7f38e406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38eb094700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38e406b830 0x7f38e406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38eb094700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f38e406ced0 0x7f38e40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38eb094700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e406d330 con 0x7f38e406b830 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38eb094700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e406ae60 con 0x7f38e406ced0 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38eb094700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e406a270 con 0x7f38e406e9f0 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38e406b830 0x7f38e406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38e406b830 0x7f38e406c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37362/0 (socket says 172.21.15.144:37362) 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 -- 172.21.15.144:0/4141595722 learned_addr learned my addr 172.21.15.144:0/4141595722 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e8e30700 1 --2- 172.21.15.144:0/4141595722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406e9f0 0x7f38e406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e9631700 1 --2- 172.21.15.144:0/4141595722 >> v2:172.21.15.177:3300/0 conn(0x7f38e406ced0 0x7f38e40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 -- 172.21.15.144:0/4141595722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406e9f0 msgr2=0x7f38e406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:53.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 --2- 172.21.15.144:0/4141595722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406e9f0 0x7f38e406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 -- 172.21.15.144:0/4141595722 >> v2:172.21.15.177:3300/0 conn(0x7f38e406ced0 msgr2=0x7f38e40ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 --2- 172.21.15.144:0/4141595722 >> v2:172.21.15.177:3300/0 conn(0x7f38e406ced0 0x7f38e40ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 -- 172.21.15.144:0/4141595722 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e406a530 con 0x7f38e406b830 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e9631700 1 --2- 172.21.15.144:0/4141595722 >> v2:172.21.15.177:3300/0 conn(0x7f38e406ced0 0x7f38e40ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.743+0000 7f38e3fff700 1 --2- 172.21.15.144:0/4141595722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406b830 0x7f38e406c990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f38e4070790 tx=0x7f38d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdb02f729d650f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38e37fe700 1 -- 172.21.15.144:0/4141595722 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000b870 con 0x7f38e406b830 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38e37fe700 1 -- 172.21.15.144:0/4141595722 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d000b9d0 con 0x7f38e406b830 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38e37fe700 1 -- 172.21.15.144:0/4141595722 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000bd60 con 0x7f38e406b830 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 -- 172.21.15.144:0/4141595722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406b830 msgr2=0x7f38e406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 --2- 172.21.15.144:0/4141595722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406b830 0x7f38e406c990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f38e4070790 tx=0x7f38d0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 -- 172.21.15.144:0/4141595722 shutdown_connections 2023-11-28T11:52:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 --2- 172.21.15.144:0/4141595722 >> v2:172.21.15.177:3300/0 conn(0x7f38e406ced0 0x7f38e40ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 --2- 172.21.15.144:0/4141595722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406e9f0 0x7f38e406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 --2- 172.21.15.144:0/4141595722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406b830 0x7f38e406c990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 -- 172.21.15.144:0/4141595722 >> 172.21.15.144:0/4141595722 conn(0x7f38e40b12b0 msgr2=0x7f38e40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 -- 172.21.15.144:0/4141595722 shutdown_connections 2023-11-28T11:52:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.744+0000 7f38eb094700 1 -- 172.21.15.144:0/4141595722 wait complete. 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.745+0000 7f38eb094700 1 Processor -- start 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.745+0000 7f38eb094700 1 -- start start 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38eb094700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406b830 0x7f38e415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38eb094700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 0x7f38e40c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38eb094700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38e406e9f0 0x7f38e40c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38eb094700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40c7d20 con 0x7f38e406ced0 2023-11-28T11:52:53.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38eb094700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40c86f0 con 0x7f38e406e9f0 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38eb094700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4164b30 con 0x7f38e406b830 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 0x7f38e40c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 0x7f38e40c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37370/0 (socket says 172.21.15.144:37370) 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38e3fff700 1 -- 172.21.15.144:0/1838498117 learned_addr learned my addr 172.21.15.144:0/1838498117 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38e8e30700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406b830 0x7f38e415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.746+0000 7f38e9631700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38e406e9f0 0x7f38e40c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e3fff700 1 -- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406b830 msgr2=0x7f38e415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e3fff700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406b830 0x7f38e415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e3fff700 1 -- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38e406e9f0 msgr2=0x7f38e40c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e3fff700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38e406e9f0 0x7f38e40c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e3fff700 1 -- 172.21.15.144:0/1838498117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0009550 con 0x7f38e406ced0 2023-11-28T11:52:53.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e3fff700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 0x7f38e40c81b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f38d0000c00 tx=0x7f38d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0004310 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d001e070 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e406aaf0 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00036a0 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.747+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e406a810 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.748+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e406edc0 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.748+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38d0029020 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.749+0000 7f38e17fa700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38c80682f0 0x7f38c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.749+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38d0086d40 con 0x7f38e406ced0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.749+0000 7f38e8e30700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38c80682f0 0x7f38c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:53.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.749+0000 7f38e8e30700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38c80682f0 0x7f38c806a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f38d80099d0 tx=0x7f38d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:53.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.753+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f38d00535a0 con 0x7f38e406ced0 2023-11-28T11:52:53.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.936+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e40c7e60 con 0x7f38e406ced0 2023-11-28T11:52:53.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.937+0000 7f38e17fa700 1 -- 172.21.15.144:0/1838498117 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+17969 (secure 0 0 0) 0x7f38d0056bc0 con 0x7f38e406ced0 2023-11-28T11:52:53.940 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:53.940 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":46,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:52:51.209427+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:52:53.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38c80682f0 msgr2=0x7f38c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38c80682f0 0x7f38c806a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f38d80099d0 tx=0x7f38d8008040 comp rx=0 tx=0).stop 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 msgr2=0x7f38e40c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 0x7f38e40c81b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f38d0000c00 tx=0x7f38d0004750 comp rx=0 tx=0).stop 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 shutdown_connections 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f38c80682f0 0x7f38c806a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38e406e9f0 0x7f38e40c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38e406b830 0x7f38e415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 --2- 172.21.15.144:0/1838498117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e406ced0 0x7f38e40c81b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 >> 172.21.15.144:0/1838498117 conn(0x7f38e40b12b0 msgr2=0x7f38e40ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:53.942 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 shutdown_connections 2023-11-28T11:52:53.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:53.940+0000 7f38eb094700 1 -- 172.21.15.144:0/1838498117 wait complete. 2023-11-28T11:52:53.954 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:52:53.954 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:54.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.072+0000 7f8b835b5700 1 Processor -- start 2023-11-28T11:52:54.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.072+0000 7f8b835b5700 1 -- start start 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.073+0000 7f8b835b5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8b7c071040 0x7f8b7c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.073+0000 7f8b835b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.073+0000 7f8b835b5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c071950 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.073+0000 7f8b835b5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c161550 con 0x7f8b7c071040 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.073+0000 7f8b835b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15d060 con 0x7f8b7c06dee0 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:49460/0 (socket says 172.21.15.144:49460) 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 -- 172.21.15.144:0/2812310974 learned_addr learned my addr 172.21.15.144:0/2812310974 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:54.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b81351700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8b7c071040 0x7f8b7c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 -- 172.21.15.144:0/2812310974 >> v2:172.21.15.177:3300/0 conn(0x7f8b7c071040 msgr2=0x7f8b7c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 --2- 172.21.15.144:0/2812310974 >> v2:172.21.15.177:3300/0 conn(0x7f8b7c071040 0x7f8b7c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 -- 172.21.15.144:0/2812310974 >> v1:172.21.15.177:6789/0 conn(0x7f8b7c06cdc0 legacy=0x7f8b7c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 -- 172.21.15.144:0/2812310974 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c15d340 con 0x7f8b7c06dee0 2023-11-28T11:52:54.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b80b50700 1 --2- 172.21.15.144:0/2812310974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c06e330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b780098f0 tx=0x7f8b78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1484d386e6a390b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b73fff700 1 -- 172.21.15.144:0/2812310974 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78011830 con 0x7f8b7c06dee0 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b73fff700 1 -- 172.21.15.144:0/2812310974 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78011990 con 0x7f8b7c06dee0 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.074+0000 7f8b73fff700 1 -- 172.21.15.144:0/2812310974 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78011d20 con 0x7f8b7c06dee0 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 -- 172.21.15.144:0/2812310974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 msgr2=0x7f8b7c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 --2- 172.21.15.144:0/2812310974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c06e330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b780098f0 tx=0x7f8b78009c00 comp rx=0 tx=0).stop 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 -- 172.21.15.144:0/2812310974 shutdown_connections 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 --2- 172.21.15.144:0/2812310974 >> v2:172.21.15.177:3300/0 conn(0x7f8b7c071040 0x7f8b7c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 --2- 172.21.15.144:0/2812310974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c06e330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 -- 172.21.15.144:0/2812310974 >> 172.21.15.144:0/2812310974 conn(0x7f8b7c0d0000 msgr2=0x7f8b7c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 -- 172.21.15.144:0/2812310974 shutdown_connections 2023-11-28T11:52:54.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.075+0000 7f8b835b5700 1 -- 172.21.15.144:0/2812310974 wait complete. 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.076+0000 7f8b835b5700 1 Processor -- start 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.076+0000 7f8b835b5700 1 -- start start 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b835b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 0x7f8b7c0ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b835b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b835b5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c071040 0x7f8b7c0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b835b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1565e0 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.083 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b835b5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c4680 con 0x7f8b7c071040 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b835b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c4960 con 0x7f8b7c06dee0 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 0x7f8b7c0ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 0x7f8b7c0ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37382/0 (socket says 172.21.15.144:37382) 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 -- 172.21.15.144:0/666233557 learned_addr learned my addr 172.21.15.144:0/666233557 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b80b50700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81b52700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c071040 0x7f8b7c0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 -- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 msgr2=0x7f8b7c0adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c0adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 -- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c071040 msgr2=0x7f8b7c0c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.077+0000 7f8b81351700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c071040 0x7f8b7c0c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b81351700 1 -- 172.21.15.144:0/666233557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b78009550 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b81351700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 0x7f8b7c0ada10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e9b0 tx=0x7f8b6c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c009710 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c00c850 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00cbc0 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c15d900 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.078+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c15d620 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.079+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c06e870 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.079+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8b6c023070 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.080+0000 7f8b71ffb700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8b600682f0 0x7f8b6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.080+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b6c085f00 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.080+0000 7f8b80b50700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8b600682f0 0x7f8b6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.080+0000 7f8b80b50700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8b600682f0 0x7f8b6006a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b78000c00 tx=0x7f8b78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.084+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8b6c052760 con 0x7f8b7c06cdc0 2023-11-28T11:52:54.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.262+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b7c0c5570 con 0x7f8b600682f0 2023-11-28T11:52:54.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.270+0000 7f8b71ffb700 1 -- 172.21.15.144:0/666233557 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302768 (secure 0 0 0) 0x7f8b7c0c5570 con 0x7f8b600682f0 2023-11-28T11:52:54.276 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:54.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8b600682f0 msgr2=0x7f8b6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8b600682f0 0x7f8b6006a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8b78000c00 tx=0x7f8b78008030 comp rx=0 tx=0).stop 2023-11-28T11:52:54.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 msgr2=0x7f8b7c0ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 0x7f8b7c0ada10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00e9b0 tx=0x7f8b6c00ecc0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 shutdown_connections 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b7c06cdc0 0x7f8b7c0ada10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8b600682f0 0x7f8b6006a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b7c071040 0x7f8b7c0c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 --2- 172.21.15.144:0/666233557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b7c06dee0 0x7f8b7c0adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 >> 172.21.15.144:0/666233557 conn(0x7f8b7c0d0000 msgr2=0x7f8b7c0cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 shutdown_connections 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.274+0000 7f8b835b5700 1 -- 172.21.15.144:0/666233557 wait complete. 2023-11-28T11:52:54.279 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:54.288 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-28T11:52:52.502976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71988,"kb_used_data":5044,"kb_used_omap":0,"kb_used_meta":66880,"kb_avail":754902732,"statfs":{"total":773094113280,"available":773020397568,"internally_reserved":0,"allocated":5165056,"data_stored":2835378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68485120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000612"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.823962+0000","last_change":"2023-11-28T11:52:49.823962+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.823962+0000","last_undegraded":"2023-11-28T11:52:49.823962+0000","last_fullsized":"2023-11-28T11:52:49.823962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:11.475783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.824712+0000","last_change":"2023-11-28T11:52:49.824712+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.824712+0000","last_undegraded":"2023-11-28T11:52:49.824712+0000","last_fullsized":"2023-11-28T11:52:49.824712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:37.366923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822634+0000","last_change":"2023-11-28T11:52:49.822634+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822634+0000","last_undegraded":"2023-11-28T11:52:49.822634+0000","last_fullsized":"2023-11-28T11:52:49.822634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:28.225138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822447+0000","last_change":"2023-11-28T11:52:49.822447+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822447+0000","last_undegraded":"2023-11-28T11:52:49.822447+0000","last_fullsized":"2023-11-28T11:52:49.822447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.465684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822496+0000","last_change":"2023-11-28T11:52:49.822496+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822496+0000","last_undegraded":"2023-11-28T11:52:49.822496+0000","last_fullsized":"2023-11-28T11:52:49.822496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:54:06.429788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817298+0000","last_change":"2023-11-28T11:52:49.817298+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817298+0000","last_undegraded":"2023-11-28T11:52:49.817298+0000","last_fullsized":"2023-11-28T11:52:49.817298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:53.755885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817657+0000","last_change":"2023-11-28T11:52:49.817657+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817657+0000","last_undegraded":"2023-11-28T11:52:49.817657+0000","last_fullsized":"2023-11-28T11:52:49.817657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:52.523879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815808+0000","last_change":"2023-11-28T11:52:49.815808+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815808+0000","last_undegraded":"2023-11-28T11:52:49.815808+0000","last_fullsized":"2023-11-28T11:52:49.815808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:00.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815142+0000","last_change":"2023-11-28T11:52:49.815142+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815142+0000","last_undegraded":"2023-11-28T11:52:49.815142+0000","last_fullsized":"2023-11-28T11:52:49.815142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:42.026408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.809482+0000","last_change":"2023-11-28T11:52:48.809482+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.809482+0000","last_undegraded":"2023-11-28T11:52:48.809482+0000","last_fullsized":"2023-11-28T11:52:48.809482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:54.890718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813849+0000","last_change":"2023-11-28T11:52:48.813849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813849+0000","last_undegraded":"2023-11-28T11:52:48.813849+0000","last_fullsized":"2023-11-28T11:52:48.813849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813794+0000","last_change":"2023-11-28T11:52:48.813794+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813794+0000","last_undegraded":"2023-11-28T11:52:48.813794+0000","last_fullsized":"2023-11-28T11:52:48.813794+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815867+0000","last_change":"2023-11-28T11:52:48.815867+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815867+0000","last_undegraded":"2023-11-28T11:52:48.815867+0000","last_fullsized":"2023-11-28T11:52:48.815867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:02.973674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829812+0000","last_change":"2023-11-28T11:52:49.827097+0000","last_active":"2023-11-28T11:52:49.829812+0000","last_peered":"2023-11-28T11:52:49.829812+0000","last_clean":"2023-11-28T11:52:49.829812+0000","last_became_active":"2023-11-28T11:52:49.826826+0000","last_became_peered":"2023-11-28T11:52:49.826826+0000","last_unstale":"2023-11-28T11:52:49.829812+0000","last_undegraded":"2023-11-28T11:52:49.829812+0000","last_fullsized":"2023-11-28T11:52:49.829812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:23.759602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811534+0000","last_change":"2023-11-28T11:52:48.811534+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811534+0000","last_undegraded":"2023-11-28T11:52:48.811534+0000","last_fullsized":"2023-11-28T11:52:48.811534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808962+0000","last_change":"2023-11-28T11:52:48.808962+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808962+0000","last_undegraded":"2023-11-28T11:52:48.808962+0000","last_fullsized":"2023-11-28T11:52:48.808962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:22.861924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829014+0000","last_change":"2023-11-28T11:52:49.822322+0000","last_active":"2023-11-28T11:52:49.829014+0000","last_peered":"2023-11-28T11:52:49.829014+0000","last_clean":"2023-11-28T11:52:49.829014+0000","last_became_active":"2023-11-28T11:52:49.822077+0000","last_became_peered":"2023-11-28T11:52:49.822077+0000","last_unstale":"2023-11-28T11:52:49.829014+0000","last_undegraded":"2023-11-28T11:52:49.829014+0000","last_fullsized":"2023-11-28T11:52:49.829014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:24.019745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814558+0000","last_change":"2023-11-28T11:52:48.814558+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814558+0000","last_undegraded":"2023-11-28T11:52:48.814558+0000","last_fullsized":"2023-11-28T11:52:48.814558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:50.368503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.812849+0000","last_change":"2023-11-28T11:52:48.812849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.812849+0000","last_undegraded":"2023-11-28T11:52:48.812849+0000","last_fullsized":"2023-11-28T11:52:48.812849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:22.526251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814234+0000","last_change":"2023-11-28T11:52:49.814234+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814234+0000","last_undegraded":"2023-11-28T11:52:49.814234+0000","last_fullsized":"2023-11-28T11:52:49.814234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:30.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814475+0000","last_change":"2023-11-28T11:52:48.814475+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814475+0000","last_undegraded":"2023-11-28T11:52:48.814475+0000","last_fullsized":"2023-11-28T11:52:48.814475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:46.875258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810166+0000","last_change":"2023-11-28T11:52:48.810166+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810166+0000","last_undegraded":"2023-11-28T11:52:48.810166+0000","last_fullsized":"2023-11-28T11:52:48.810166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:16.135396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834400+0000","last_change":"2023-11-28T11:52:49.830260+0000","last_active":"2023-11-28T11:52:49.834400+0000","last_peered":"2023-11-28T11:52:49.834400+0000","last_clean":"2023-11-28T11:52:49.834400+0000","last_became_active":"2023-11-28T11:52:49.829977+0000","last_became_peered":"2023-11-28T11:52:49.829977+0000","last_unstale":"2023-11-28T11:52:49.834400+0000","last_undegraded":"2023-11-28T11:52:49.834400+0000","last_fullsized":"2023-11-28T11:52:49.834400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:19.733169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814190+0000","last_change":"2023-11-28T11:52:49.814190+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814190+0000","last_undegraded":"2023-11-28T11:52:49.814190+0000","last_fullsized":"2023-11-28T11:52:49.814190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:03.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813836+0000","last_change":"2023-11-28T11:52:48.813836+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813836+0000","last_undegraded":"2023-11-28T11:52:48.813836+0000","last_fullsized":"2023-11-28T11:52:48.813836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.978385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.816827+0000","last_change":"2023-11-28T11:52:49.816827+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.816827+0000","last_undegraded":"2023-11-28T11:52:49.816827+0000","last_fullsized":"2023-11-28T11:52:49.816827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:30.533622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:44.118206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829400+0000","last_change":"2023-11-28T11:52:49.826554+0000","last_active":"2023-11-28T11:52:49.829400+0000","last_peered":"2023-11-28T11:52:49.829400+0000","last_clean":"2023-11-28T11:52:49.829400+0000","last_became_active":"2023-11-28T11:52:49.826299+0000","last_became_peered":"2023-11-28T11:52:49.826299+0000","last_unstale":"2023-11-28T11:52:49.829400+0000","last_undegraded":"2023-11-28T11:52:49.829400+0000","last_fullsized":"2023-11-28T11:52:49.829400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:33.111523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813679+0000","last_change":"2023-11-28T11:52:48.813679+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813679+0000","last_undegraded":"2023-11-28T11:52:48.813679+0000","last_fullsized":"2023-11-28T11:52:48.813679+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:51.984789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808979+0000","last_change":"2023-11-28T11:52:48.808979+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808979+0000","last_undegraded":"2023-11-28T11:52:48.808979+0000","last_fullsized":"2023-11-28T11:52:48.808979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:23.803406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813777+0000","last_change":"2023-11-28T11:52:48.813777+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813777+0000","last_undegraded":"2023-11-28T11:52:48.813777+0000","last_fullsized":"2023-11-28T11:52:48.813777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810666+0000","last_change":"2023-11-28T11:52:48.810666+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810666+0000","last_undegraded":"2023-11-28T11:52:48.810666+0000","last_fullsized":"2023-11-28T11:52:48.810666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:54.122734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814746+0000","last_change":"2023-11-28T11:52:49.814746+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814746+0000","last_undegraded":"2023-11-28T11:52:49.814746+0000","last_fullsized":"2023-11-28T11:52:49.814746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:26.868306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834375+0000","last_change":"2023-11-28T11:52:49.829671+0000","last_active":"2023-11-28T11:52:49.834375+0000","last_peered":"2023-11-28T11:52:49.834375+0000","last_clean":"2023-11-28T11:52:49.834375+0000","last_became_active":"2023-11-28T11:52:49.829404+0000","last_became_peered":"2023-11-28T11:52:49.829404+0000","last_unstale":"2023-11-28T11:52:49.834375+0000","last_undegraded":"2023-11-28T11:52:49.834375+0000","last_fullsized":"2023-11-28T11:52:49.834375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:18.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813638+0000","last_change":"2023-11-28T11:52:48.813638+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813638+0000","last_undegraded":"2023-11-28T11:52:48.813638+0000","last_fullsized":"2023-11-28T11:52:48.813638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:57.828731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814357+0000","last_change":"2023-11-28T11:52:49.814357+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814357+0000","last_undegraded":"2023-11-28T11:52:49.814357+0000","last_fullsized":"2023-11-28T11:52:49.814357+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:04.510511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810139+0000","last_change":"2023-11-28T11:52:48.810139+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810139+0000","last_undegraded":"2023-11-28T11:52:48.810139+0000","last_fullsized":"2023-11-28T11:52:48.810139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:39.463589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286150+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:46.286150+0000","last_peered":"2023-11-28T11:52:46.286150+0000","last_clean":"2023-11-28T11:52:46.286150+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:46.286150+0000","last_undegraded":"2023-11-28T11:52:46.286150+0000","last_fullsized":"2023-11-28T11:52:46.286150+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:33.980547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132093+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:46.132093+0000","last_peered":"2023-11-28T11:52:46.132093+0000","last_clean":"2023-11-28T11:52:46.132093+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:46.132093+0000","last_undegraded":"2023-11-28T11:52:46.132093+0000","last_fullsized":"2023-11-28T11:52:46.132093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813768+0000","last_change":"2023-11-28T11:52:48.813768+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813768+0000","last_undegraded":"2023-11-28T11:52:48.813768+0000","last_fullsized":"2023-11-28T11:52:48.813768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:34.991301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.285986+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:46.285986+0000","last_peered":"2023-11-28T11:52:46.285986+0000","last_clean":"2023-11-28T11:52:46.285986+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:46.285986+0000","last_undegraded":"2023-11-28T11:52:46.285986+0000","last_fullsized":"2023-11-28T11:52:46.285986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":181,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132872+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:46.132872+0000","last_peered":"2023-11-28T11:52:46.132872+0000","last_clean":"2023-11-28T11:52:46.132872+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:46.132872+0000","last_undegraded":"2023-11-28T11:52:46.132872+0000","last_fullsized":"2023-11-28T11:52:46.132872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829752+0000","last_change":"2023-11-28T11:52:49.824383+0000","last_active":"2023-11-28T11:52:49.829752+0000","last_peered":"2023-11-28T11:52:49.829752+0000","last_clean":"2023-11-28T11:52:49.829752+0000","last_became_active":"2023-11-28T11:52:49.824081+0000","last_became_peered":"2023-11-28T11:52:49.824081+0000","last_unstale":"2023-11-28T11:52:49.829752+0000","last_undegraded":"2023-11-28T11:52:49.829752+0000","last_fullsized":"2023-11-28T11:52:49.829752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:17.464586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132870+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:46.132870+0000","last_peered":"2023-11-28T11:52:46.132870+0000","last_clean":"2023-11-28T11:52:46.132870+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:46.132870+0000","last_undegraded":"2023-11-28T11:52:46.132870+0000","last_fullsized":"2023-11-28T11:52:46.132870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834419+0000","last_change":"2023-11-28T11:52:49.830269+0000","last_active":"2023-11-28T11:52:49.834419+0000","last_peered":"2023-11-28T11:52:49.834419+0000","last_clean":"2023-11-28T11:52:49.834419+0000","last_became_active":"2023-11-28T11:52:49.830011+0000","last_became_peered":"2023-11-28T11:52:49.830011+0000","last_unstale":"2023-11-28T11:52:49.834419+0000","last_undegraded":"2023-11-28T11:52:49.834419+0000","last_fullsized":"2023-11-28T11:52:49.834419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:05.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132859+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:46.132859+0000","last_peered":"2023-11-28T11:52:46.132859+0000","last_clean":"2023-11-28T11:52:46.132859+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:46.132859+0000","last_undegraded":"2023-11-28T11:52:46.132859+0000","last_fullsized":"2023-11-28T11:52:46.132859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808847+0000","last_change":"2023-11-28T11:52:48.808847+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808847+0000","last_undegraded":"2023-11-28T11:52:48.808847+0000","last_fullsized":"2023-11-28T11:52:48.808847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286133+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:46.286133+0000","last_peered":"2023-11-28T11:52:46.286133+0000","last_clean":"2023-11-28T11:52:46.286133+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:46.286133+0000","last_undegraded":"2023-11-28T11:52:46.286133+0000","last_fullsized":"2023-11-28T11:52:46.286133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815863+0000","last_change":"2023-11-28T11:52:48.815863+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815863+0000","last_undegraded":"2023-11-28T11:52:48.815863+0000","last_fullsized":"2023-11-28T11:52:48.815863+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131458+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:46.131458+0000","last_peered":"2023-11-28T11:52:46.131458+0000","last_clean":"2023-11-28T11:52:46.131458+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:46.131458+0000","last_undegraded":"2023-11-28T11:52:46.131458+0000","last_fullsized":"2023-11-28T11:52:46.131458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815176+0000","last_change":"2023-11-28T11:52:49.815176+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815176+0000","last_undegraded":"2023-11-28T11:52:49.815176+0000","last_fullsized":"2023-11-28T11:52:49.815176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:55.616287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811601+0000","last_change":"2023-11-28T11:52:48.811601+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811601+0000","last_undegraded":"2023-11-28T11:52:48.811601+0000","last_fullsized":"2023-11-28T11:52:48.811601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:48:34.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131493+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:46.131493+0000","last_peered":"2023-11-28T11:52:46.131493+0000","last_clean":"2023-11-28T11:52:46.131493+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:46.131493+0000","last_undegraded":"2023-11-28T11:52:46.131493+0000","last_fullsized":"2023-11-28T11:52:46.131493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9896,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361944,"statfs":{"total":96636764160,"available":96626630656,"internally_reserved":0,"allocated":1081344,"data_stored":792046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59899999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":3.7810000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9716,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362124,"statfs":{"total":96636764160,"available":96626814976,"internally_reserved":0,"allocated":1093632,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.30199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75700000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8684,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363156,"statfs":{"total":96636764160,"available":96627871744,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.32900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.317}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.501}]}]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8880,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362960,"statfs":{"total":96636764160,"available":96627671040,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.254},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.30199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":16.231000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.755},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.441},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:54.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:54.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.447+0000 7fc6987a9700 1 Processor -- start 2023-11-28T11:52:54.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.447+0000 7fc6987a9700 1 -- start start 2023-11-28T11:52:54.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.448+0000 7fc6987a9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc6900b3040 0x7fc69015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.448+0000 7fc6987a9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69006b420 con 0x7fc6900d0a00 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.448+0000 7fc6987a9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69006b700 con 0x7fc6900ce040 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.448+0000 7fc6987a9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69006ff60 con 0x7fc6900b3040 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.448+0000 7fc695d44700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc6900d0a00 0x7fc6900b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54104/0 (socket says 172.21.15.144:54104) 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 -- 172.21.15.144:0/1136359067 learned_addr learned my addr 172.21.15.144:0/1136359067 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399324413 0 0) 0x7fc69006b420 con 0x7fc6900d0a00 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc696d46700 1 --2- 172.21.15.144:0/1136359067 >> v2:172.21.15.177:3300/0 conn(0x7fc6900b3040 0x7fc69015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c0035b0 con 0x7fc6900d0a00 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 151370766 0 0) 0x7fc67c0035b0 con 0x7fc6900d0a00 2023-11-28T11:52:54.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 >> v2:172.21.15.177:3300/0 conn(0x7fc6900b3040 msgr2=0x7fc69015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 --2- 172.21.15.144:0/1136359067 >> v2:172.21.15.177:3300/0 conn(0x7fc6900b3040 0x7fc69015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 >> v1:172.21.15.177:6789/0 conn(0x7fc6900ce040 legacy=0x7fc69006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69006b9e0 con 0x7fc6900d0a00 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fc68c001670 con 0x7fc6900d0a00 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 >> v1:172.21.15.144:6789/0 conn(0x7fc6900d0a00 legacy=0x7fc6900b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc67c003a00 0x7fc67c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004850 0x7fc67c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c005200 0x7fc67c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69006ff60 con 0x7fc67c004850 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c00c080 con 0x7fc67c005200 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c00c240 con 0x7fc67c003a00 2023-11-28T11:52:54.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004850 0x7fc67c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc696545700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc67c003a00 0x7fc67c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc696d46700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c005200 0x7fc67c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 -- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc67c003a00 msgr2=0x7fc67c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc67c003a00 0x7fc67c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 -- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c005200 msgr2=0x7fc67c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c005200 0x7fc67c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc695d44700 1 -- 172.21.15.144:0/1136359067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c0035b0 con 0x7fc67c004850 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc696d46700 1 -- 172.21.15.144:0/1136359067 reap_dead start 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.449+0000 7fc696545700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc67c003a00 0x7fc67c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc695d44700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004850 0x7fc67c004cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c240 tx=0x7fc67c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7892af81ea60cba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c007c10 con 0x7fc67c004850 2023-11-28T11:52:54.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00cd60 con 0x7fc67c004850 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc695543700 1 -- 172.21.15.144:0/1136359067 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c005ba0 con 0x7fc67c004850 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 -- 172.21.15.144:0/1136359067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004850 msgr2=0x7fc67c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004850 0x7fc67c004cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c240 tx=0x7fc67c005880 comp rx=0 tx=0).stop 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 -- 172.21.15.144:0/1136359067 shutdown_connections 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 --2- 172.21.15.144:0/1136359067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004850 0x7fc67c004cc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 -- 172.21.15.144:0/1136359067 >> 172.21.15.144:0/1136359067 conn(0x7fc69006aed0 msgr2=0x7fc69006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 -- 172.21.15.144:0/1136359067 shutdown_connections 2023-11-28T11:52:54.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.450+0000 7fc6987a9700 1 -- 172.21.15.144:0/1136359067 wait complete. 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 Processor -- start 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 -- start start 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c004850 0x7fc690150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 0x7fc6900cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc69015a5b0 0x7fc69015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900cc360 con 0x7fc6900cd400 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900cc640 con 0x7fc67c004850 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc6987a9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900cc920 con 0x7fc69015a5b0 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc695d44700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 0x7fc6900cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc695d44700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 0x7fc6900cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37406/0 (socket says 172.21.15.144:37406) 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc695d44700 1 -- 172.21.15.144:0/2420027601 learned_addr learned my addr 172.21.15.144:0/2420027601 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:54.459 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc696d46700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc69015a5b0 0x7fc69015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.452+0000 7fc696545700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c004850 0x7fc690150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc695d44700 1 -- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc69015a5b0 msgr2=0x7fc69015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc695d44700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc69015a5b0 0x7fc69015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc695d44700 1 -- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c004850 msgr2=0x7fc690150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc695d44700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c004850 0x7fc690150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc695d44700 1 -- 172.21.15.144:0/2420027601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69006bfa0 con 0x7fc6900cd400 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc695d44700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 0x7fc6900cd870 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c830 tx=0x7fc68c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00ced0 con 0x7fc6900cd400 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c001640 con 0x7fc6900cd400 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0072d0 con 0x7fc6900cd400 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69006bcc0 con 0x7fc6900cd400 2023-11-28T11:52:54.460 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.453+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69015df10 con 0x7fc6900cd400 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.454+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6900cd250 con 0x7fc6900cd400 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.454+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc68c024030 con 0x7fc6900cd400 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.455+0000 7fc682ffd700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6740682f0 0x7fc67406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.455+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc68c086f60 con 0x7fc6900cd400 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.455+0000 7fc696545700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6740682f0 0x7fc67406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.455+0000 7fc696545700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6740682f0 0x7fc67406a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc684000c00 tx=0x7fc684009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.461 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.459+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc68c0b85e0 con 0x7fc6900cd400 2023-11-28T11:52:54.638 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.636+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6900cc7b0 con 0x7fc6740682f0 2023-11-28T11:52:54.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.644+0000 7fc682ffd700 1 -- 172.21.15.144:0/2420027601 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302768 (secure 0 0 0) 0x7fc6900cc7b0 con 0x7fc6740682f0 2023-11-28T11:52:54.650 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:54.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6740682f0 msgr2=0x7fc67406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6740682f0 0x7fc67406a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc684000c00 tx=0x7fc684009040 comp rx=0 tx=0).stop 2023-11-28T11:52:54.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 msgr2=0x7fc6900cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 0x7fc6900cd870 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c830 tx=0x7fc68c001a00 comp rx=0 tx=0).stop 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 shutdown_connections 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6740682f0 0x7fc67406a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc69015a5b0 0x7fc69015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c004850 0x7fc690150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 --2- 172.21.15.144:0/2420027601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6900cd400 0x7fc6900cd870 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 >> 172.21.15.144:0/2420027601 conn(0x7fc69006aed0 msgr2=0x7fc6901e7380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 shutdown_connections 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.647+0000 7fc6987a9700 1 -- 172.21.15.144:0/2420027601 wait complete. 2023-11-28T11:52:54.653 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:54.661 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-28T11:52:54.503286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71988,"kb_used_data":5044,"kb_used_omap":0,"kb_used_meta":66880,"kb_avail":754902732,"statfs":{"total":773094113280,"available":773020397568,"internally_reserved":0,"allocated":5165056,"data_stored":2835378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68485120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.695937"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.823962+0000","last_change":"2023-11-28T11:52:49.823962+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.823962+0000","last_undegraded":"2023-11-28T11:52:49.823962+0000","last_fullsized":"2023-11-28T11:52:49.823962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:11.475783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.824712+0000","last_change":"2023-11-28T11:52:49.824712+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.824712+0000","last_undegraded":"2023-11-28T11:52:49.824712+0000","last_fullsized":"2023-11-28T11:52:49.824712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:37.366923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822634+0000","last_change":"2023-11-28T11:52:49.822634+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822634+0000","last_undegraded":"2023-11-28T11:52:49.822634+0000","last_fullsized":"2023-11-28T11:52:49.822634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:28.225138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822447+0000","last_change":"2023-11-28T11:52:49.822447+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822447+0000","last_undegraded":"2023-11-28T11:52:49.822447+0000","last_fullsized":"2023-11-28T11:52:49.822447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.465684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822496+0000","last_change":"2023-11-28T11:52:49.822496+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822496+0000","last_undegraded":"2023-11-28T11:52:49.822496+0000","last_fullsized":"2023-11-28T11:52:49.822496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:54:06.429788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817298+0000","last_change":"2023-11-28T11:52:49.817298+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817298+0000","last_undegraded":"2023-11-28T11:52:49.817298+0000","last_fullsized":"2023-11-28T11:52:49.817298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:53.755885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817657+0000","last_change":"2023-11-28T11:52:49.817657+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817657+0000","last_undegraded":"2023-11-28T11:52:49.817657+0000","last_fullsized":"2023-11-28T11:52:49.817657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:52.523879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815808+0000","last_change":"2023-11-28T11:52:49.815808+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815808+0000","last_undegraded":"2023-11-28T11:52:49.815808+0000","last_fullsized":"2023-11-28T11:52:49.815808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:00.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815142+0000","last_change":"2023-11-28T11:52:49.815142+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815142+0000","last_undegraded":"2023-11-28T11:52:49.815142+0000","last_fullsized":"2023-11-28T11:52:49.815142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:42.026408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.809482+0000","last_change":"2023-11-28T11:52:48.809482+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.809482+0000","last_undegraded":"2023-11-28T11:52:48.809482+0000","last_fullsized":"2023-11-28T11:52:48.809482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:54.890718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813849+0000","last_change":"2023-11-28T11:52:48.813849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813849+0000","last_undegraded":"2023-11-28T11:52:48.813849+0000","last_fullsized":"2023-11-28T11:52:48.813849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813794+0000","last_change":"2023-11-28T11:52:48.813794+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813794+0000","last_undegraded":"2023-11-28T11:52:48.813794+0000","last_fullsized":"2023-11-28T11:52:48.813794+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815867+0000","last_change":"2023-11-28T11:52:48.815867+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815867+0000","last_undegraded":"2023-11-28T11:52:48.815867+0000","last_fullsized":"2023-11-28T11:52:48.815867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:02.973674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829812+0000","last_change":"2023-11-28T11:52:49.827097+0000","last_active":"2023-11-28T11:52:49.829812+0000","last_peered":"2023-11-28T11:52:49.829812+0000","last_clean":"2023-11-28T11:52:49.829812+0000","last_became_active":"2023-11-28T11:52:49.826826+0000","last_became_peered":"2023-11-28T11:52:49.826826+0000","last_unstale":"2023-11-28T11:52:49.829812+0000","last_undegraded":"2023-11-28T11:52:49.829812+0000","last_fullsized":"2023-11-28T11:52:49.829812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:23.759602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811534+0000","last_change":"2023-11-28T11:52:48.811534+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811534+0000","last_undegraded":"2023-11-28T11:52:48.811534+0000","last_fullsized":"2023-11-28T11:52:48.811534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808962+0000","last_change":"2023-11-28T11:52:48.808962+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808962+0000","last_undegraded":"2023-11-28T11:52:48.808962+0000","last_fullsized":"2023-11-28T11:52:48.808962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:22.861924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829014+0000","last_change":"2023-11-28T11:52:49.822322+0000","last_active":"2023-11-28T11:52:49.829014+0000","last_peered":"2023-11-28T11:52:49.829014+0000","last_clean":"2023-11-28T11:52:49.829014+0000","last_became_active":"2023-11-28T11:52:49.822077+0000","last_became_peered":"2023-11-28T11:52:49.822077+0000","last_unstale":"2023-11-28T11:52:49.829014+0000","last_undegraded":"2023-11-28T11:52:49.829014+0000","last_fullsized":"2023-11-28T11:52:49.829014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:24.019745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814558+0000","last_change":"2023-11-28T11:52:48.814558+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814558+0000","last_undegraded":"2023-11-28T11:52:48.814558+0000","last_fullsized":"2023-11-28T11:52:48.814558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:50.368503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.812849+0000","last_change":"2023-11-28T11:52:48.812849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.812849+0000","last_undegraded":"2023-11-28T11:52:48.812849+0000","last_fullsized":"2023-11-28T11:52:48.812849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:22.526251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814234+0000","last_change":"2023-11-28T11:52:49.814234+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814234+0000","last_undegraded":"2023-11-28T11:52:49.814234+0000","last_fullsized":"2023-11-28T11:52:49.814234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:30.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814475+0000","last_change":"2023-11-28T11:52:48.814475+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814475+0000","last_undegraded":"2023-11-28T11:52:48.814475+0000","last_fullsized":"2023-11-28T11:52:48.814475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:46.875258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810166+0000","last_change":"2023-11-28T11:52:48.810166+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810166+0000","last_undegraded":"2023-11-28T11:52:48.810166+0000","last_fullsized":"2023-11-28T11:52:48.810166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:16.135396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834400+0000","last_change":"2023-11-28T11:52:49.830260+0000","last_active":"2023-11-28T11:52:49.834400+0000","last_peered":"2023-11-28T11:52:49.834400+0000","last_clean":"2023-11-28T11:52:49.834400+0000","last_became_active":"2023-11-28T11:52:49.829977+0000","last_became_peered":"2023-11-28T11:52:49.829977+0000","last_unstale":"2023-11-28T11:52:49.834400+0000","last_undegraded":"2023-11-28T11:52:49.834400+0000","last_fullsized":"2023-11-28T11:52:49.834400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:19.733169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814190+0000","last_change":"2023-11-28T11:52:49.814190+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814190+0000","last_undegraded":"2023-11-28T11:52:49.814190+0000","last_fullsized":"2023-11-28T11:52:49.814190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:03.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813836+0000","last_change":"2023-11-28T11:52:48.813836+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813836+0000","last_undegraded":"2023-11-28T11:52:48.813836+0000","last_fullsized":"2023-11-28T11:52:48.813836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.978385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.816827+0000","last_change":"2023-11-28T11:52:49.816827+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.816827+0000","last_undegraded":"2023-11-28T11:52:49.816827+0000","last_fullsized":"2023-11-28T11:52:49.816827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:30.533622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:44.118206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829400+0000","last_change":"2023-11-28T11:52:49.826554+0000","last_active":"2023-11-28T11:52:49.829400+0000","last_peered":"2023-11-28T11:52:49.829400+0000","last_clean":"2023-11-28T11:52:49.829400+0000","last_became_active":"2023-11-28T11:52:49.826299+0000","last_became_peered":"2023-11-28T11:52:49.826299+0000","last_unstale":"2023-11-28T11:52:49.829400+0000","last_undegraded":"2023-11-28T11:52:49.829400+0000","last_fullsized":"2023-11-28T11:52:49.829400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:33.111523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813679+0000","last_change":"2023-11-28T11:52:48.813679+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813679+0000","last_undegraded":"2023-11-28T11:52:48.813679+0000","last_fullsized":"2023-11-28T11:52:48.813679+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:51.984789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808979+0000","last_change":"2023-11-28T11:52:48.808979+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808979+0000","last_undegraded":"2023-11-28T11:52:48.808979+0000","last_fullsized":"2023-11-28T11:52:48.808979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:23.803406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813777+0000","last_change":"2023-11-28T11:52:48.813777+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813777+0000","last_undegraded":"2023-11-28T11:52:48.813777+0000","last_fullsized":"2023-11-28T11:52:48.813777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810666+0000","last_change":"2023-11-28T11:52:48.810666+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810666+0000","last_undegraded":"2023-11-28T11:52:48.810666+0000","last_fullsized":"2023-11-28T11:52:48.810666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:54.122734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814746+0000","last_change":"2023-11-28T11:52:49.814746+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814746+0000","last_undegraded":"2023-11-28T11:52:49.814746+0000","last_fullsized":"2023-11-28T11:52:49.814746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:26.868306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834375+0000","last_change":"2023-11-28T11:52:49.829671+0000","last_active":"2023-11-28T11:52:49.834375+0000","last_peered":"2023-11-28T11:52:49.834375+0000","last_clean":"2023-11-28T11:52:49.834375+0000","last_became_active":"2023-11-28T11:52:49.829404+0000","last_became_peered":"2023-11-28T11:52:49.829404+0000","last_unstale":"2023-11-28T11:52:49.834375+0000","last_undegraded":"2023-11-28T11:52:49.834375+0000","last_fullsized":"2023-11-28T11:52:49.834375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:18.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813638+0000","last_change":"2023-11-28T11:52:48.813638+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813638+0000","last_undegraded":"2023-11-28T11:52:48.813638+0000","last_fullsized":"2023-11-28T11:52:48.813638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:57.828731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814357+0000","last_change":"2023-11-28T11:52:49.814357+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814357+0000","last_undegraded":"2023-11-28T11:52:49.814357+0000","last_fullsized":"2023-11-28T11:52:49.814357+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:04.510511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810139+0000","last_change":"2023-11-28T11:52:48.810139+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810139+0000","last_undegraded":"2023-11-28T11:52:48.810139+0000","last_fullsized":"2023-11-28T11:52:48.810139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:39.463589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286150+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:46.286150+0000","last_peered":"2023-11-28T11:52:46.286150+0000","last_clean":"2023-11-28T11:52:46.286150+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:46.286150+0000","last_undegraded":"2023-11-28T11:52:46.286150+0000","last_fullsized":"2023-11-28T11:52:46.286150+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:33.980547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132093+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:46.132093+0000","last_peered":"2023-11-28T11:52:46.132093+0000","last_clean":"2023-11-28T11:52:46.132093+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:46.132093+0000","last_undegraded":"2023-11-28T11:52:46.132093+0000","last_fullsized":"2023-11-28T11:52:46.132093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813768+0000","last_change":"2023-11-28T11:52:48.813768+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813768+0000","last_undegraded":"2023-11-28T11:52:48.813768+0000","last_fullsized":"2023-11-28T11:52:48.813768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:34.991301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.285986+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:46.285986+0000","last_peered":"2023-11-28T11:52:46.285986+0000","last_clean":"2023-11-28T11:52:46.285986+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:46.285986+0000","last_undegraded":"2023-11-28T11:52:46.285986+0000","last_fullsized":"2023-11-28T11:52:46.285986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":181,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132872+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:46.132872+0000","last_peered":"2023-11-28T11:52:46.132872+0000","last_clean":"2023-11-28T11:52:46.132872+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:46.132872+0000","last_undegraded":"2023-11-28T11:52:46.132872+0000","last_fullsized":"2023-11-28T11:52:46.132872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829752+0000","last_change":"2023-11-28T11:52:49.824383+0000","last_active":"2023-11-28T11:52:49.829752+0000","last_peered":"2023-11-28T11:52:49.829752+0000","last_clean":"2023-11-28T11:52:49.829752+0000","last_became_active":"2023-11-28T11:52:49.824081+0000","last_became_peered":"2023-11-28T11:52:49.824081+0000","last_unstale":"2023-11-28T11:52:49.829752+0000","last_undegraded":"2023-11-28T11:52:49.829752+0000","last_fullsized":"2023-11-28T11:52:49.829752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:17.464586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132870+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:46.132870+0000","last_peered":"2023-11-28T11:52:46.132870+0000","last_clean":"2023-11-28T11:52:46.132870+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:46.132870+0000","last_undegraded":"2023-11-28T11:52:46.132870+0000","last_fullsized":"2023-11-28T11:52:46.132870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834419+0000","last_change":"2023-11-28T11:52:49.830269+0000","last_active":"2023-11-28T11:52:49.834419+0000","last_peered":"2023-11-28T11:52:49.834419+0000","last_clean":"2023-11-28T11:52:49.834419+0000","last_became_active":"2023-11-28T11:52:49.830011+0000","last_became_peered":"2023-11-28T11:52:49.830011+0000","last_unstale":"2023-11-28T11:52:49.834419+0000","last_undegraded":"2023-11-28T11:52:49.834419+0000","last_fullsized":"2023-11-28T11:52:49.834419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:05.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132859+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:46.132859+0000","last_peered":"2023-11-28T11:52:46.132859+0000","last_clean":"2023-11-28T11:52:46.132859+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:46.132859+0000","last_undegraded":"2023-11-28T11:52:46.132859+0000","last_fullsized":"2023-11-28T11:52:46.132859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808847+0000","last_change":"2023-11-28T11:52:48.808847+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808847+0000","last_undegraded":"2023-11-28T11:52:48.808847+0000","last_fullsized":"2023-11-28T11:52:48.808847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286133+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:46.286133+0000","last_peered":"2023-11-28T11:52:46.286133+0000","last_clean":"2023-11-28T11:52:46.286133+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:46.286133+0000","last_undegraded":"2023-11-28T11:52:46.286133+0000","last_fullsized":"2023-11-28T11:52:46.286133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815863+0000","last_change":"2023-11-28T11:52:48.815863+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815863+0000","last_undegraded":"2023-11-28T11:52:48.815863+0000","last_fullsized":"2023-11-28T11:52:48.815863+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131458+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:46.131458+0000","last_peered":"2023-11-28T11:52:46.131458+0000","last_clean":"2023-11-28T11:52:46.131458+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:46.131458+0000","last_undegraded":"2023-11-28T11:52:46.131458+0000","last_fullsized":"2023-11-28T11:52:46.131458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815176+0000","last_change":"2023-11-28T11:52:49.815176+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815176+0000","last_undegraded":"2023-11-28T11:52:49.815176+0000","last_fullsized":"2023-11-28T11:52:49.815176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:55.616287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811601+0000","last_change":"2023-11-28T11:52:48.811601+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811601+0000","last_undegraded":"2023-11-28T11:52:48.811601+0000","last_fullsized":"2023-11-28T11:52:48.811601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:48:34.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131493+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:46.131493+0000","last_peered":"2023-11-28T11:52:46.131493+0000","last_clean":"2023-11-28T11:52:46.131493+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:46.131493+0000","last_undegraded":"2023-11-28T11:52:46.131493+0000","last_fullsized":"2023-11-28T11:52:46.131493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9896,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361944,"statfs":{"total":96636764160,"available":96626630656,"internally_reserved":0,"allocated":1081344,"data_stored":792046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59899999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":3.7810000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9716,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362124,"statfs":{"total":96636764160,"available":96626814976,"internally_reserved":0,"allocated":1093632,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.30199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75700000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8684,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363156,"statfs":{"total":96636764160,"available":96627871744,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.32900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.317}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.501}]}]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8880,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362960,"statfs":{"total":96636764160,"available":96627671040,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.254},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.30199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":16.231000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.755},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.441},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:54.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:54.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.820+0000 7f3bdc9d4700 1 Processor -- start 2023-11-28T11:52:54.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.820+0000 7f3bdc9d4700 1 -- start start 2023-11-28T11:52:54.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bdc9d4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3bd815cef0 0x7f3bd80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bdc9d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 0x7f3bd80ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bdc9d4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd80cc5f0 con 0x7f3bd80cbdf0 2023-11-28T11:52:54.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bdc9d4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd815d470 con 0x7f3bd815cef0 2023-11-28T11:52:54.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bdc9d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd815d750 con 0x7f3bd806ba90 2023-11-28T11:52:54.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd5d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3bd80cbdf0 0x7f3bd806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54124/0 (socket says 172.21.15.144:54124) 2023-11-28T11:52:54.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/2938737091 learned_addr learned my addr 172.21.15.144:0/2938737091 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:54.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd6d9d700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 0x7f3bd80ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd659c700 1 --2- 172.21.15.144:0/2938737091 >> v2:172.21.15.177:3300/0 conn(0x7f3bd815cef0 0x7f3bd80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283556749 0 0) 0x7f3bd80cc5f0 con 0x7f3bd80cbdf0 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc00035b0 con 0x7f3bd80cbdf0 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449792466 0 0) 0x7f3bc00035b0 con 0x7f3bd80cbdf0 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 msgr2=0x7f3bd80ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 0x7f3bd80ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 >> v2:172.21.15.177:3300/0 conn(0x7f3bd815cef0 msgr2=0x7f3bd80af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 --2- 172.21.15.144:0/2938737091 >> v2:172.21.15.177:3300/0 conn(0x7f3bd815cef0 0x7f3bd80af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd815da30 con 0x7f3bd80cbdf0 2023-11-28T11:52:54.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f3bcc001670 con 0x7f3bd80cbdf0 2023-11-28T11:52:54.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 >> v1:172.21.15.144:6789/0 conn(0x7f3bd80cbdf0 legacy=0x7f3bd806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.821+0000 7f3bd559a700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc0003a00 0x7f3bc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bc00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 0x7f3bc0005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd815d470 con 0x7f3bc0004430 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd815d750 con 0x7f3bc0003a00 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc000ba10 con 0x7f3bd806ba90 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bc00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd659c700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc0003a00 0x7f3bc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd6d9d700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 0x7f3bc0005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 msgr2=0x7f3bc0005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd806ba90 0x7f3bc0005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc0003a00 msgr2=0x7f3bc0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:54.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc0003a00 0x7f3bc0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 -- 172.21.15.144:0/2938737091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc00035b0 con 0x7f3bc0004430 2023-11-28T11:52:54.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bc00048f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00c270 tx=0x7f3bc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406a5c788c11ad01 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc007c10 con 0x7f3bc0004430 2023-11-28T11:52:54.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00cdd0 con 0x7f3bc0004430 2023-11-28T11:52:54.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.822+0000 7f3bd559a700 1 -- 172.21.15.144:0/2938737091 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc005ba0 con 0x7f3bc0004430 2023-11-28T11:52:54.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/2938737091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 msgr2=0x7f3bc00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/2938737091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bc00048f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00c270 tx=0x7f3bc0005880 comp rx=0 tx=0).stop 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bd6d9d700 1 -- 172.21.15.144:0/2938737091 reap_dead start 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/2938737091 shutdown_connections 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/2938737091 >> 172.21.15.144:0/2938737091 conn(0x7f3bd80d0000 msgr2=0x7f3bd80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/2938737091 shutdown_connections 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.823+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/2938737091 wait complete. 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.824+0000 7f3bdc9d4700 1 Processor -- start 2023-11-28T11:52:54.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.824+0000 7f3bdc9d4700 1 -- start start 2023-11-28T11:52:54.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bdc9d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bd81efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bdc9d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd814ec40 0x7f3bd8071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bdc9d4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071ac0 0x7f3bd80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bdc9d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80c8050 con 0x7f3bc0004430 2023-11-28T11:52:54.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bdc9d4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81f0080 con 0x7f3bd8071ac0 2023-11-28T11:52:54.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bdc9d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8071f60 con 0x7f3bd814ec40 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bd81efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bd81efb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37422/0 (socket says 172.21.15.144:37422) 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 -- 172.21.15.144:0/338163937 learned_addr learned my addr 172.21.15.144:0/338163937 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd6d9d700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071ac0 0x7f3bd80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd814ec40 0x7f3bd8071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 -- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd814ec40 msgr2=0x7f3bd8071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd814ec40 0x7f3bd8071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 -- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071ac0 msgr2=0x7f3bd80c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071ac0 0x7f3bd80c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd659c700 1 -- 172.21.15.144:0/338163937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd815dff0 con 0x7f3bc0004430 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.825+0000 7f3bd6d9d700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071ac0 0x7f3bd80c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.826+0000 7f3bd659c700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bd81efb40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8011aa0 tx=0x7f3bc8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.826+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800f920 con 0x7f3bc0004430 2023-11-28T11:52:54.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.826+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800faa0 con 0x7f3bc0004430 2023-11-28T11:52:54.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.826+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800fe50 con 0x7f3bc0004430 2023-11-28T11:52:54.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.826+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd815dd10 con 0x7f3bc0004430 2023-11-28T11:52:54.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.826+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd80ca650 con 0x7f3bc0004430 2023-11-28T11:52:54.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.827+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd80d0e60 con 0x7f3bc0004430 2023-11-28T11:52:54.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.827+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3bc8014440 con 0x7f3bc0004430 2023-11-28T11:52:54.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.827+0000 7f3bc6ffd700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:54.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.827+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3bc8086510 con 0x7f3bc0004430 2023-11-28T11:52:54.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.827+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:54.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.828+0000 7f3bd5d9b700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8071070 tx=0x7f3bcc007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:54.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:54.832+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3bc8052dd0 con 0x7f3bc0004430 2023-11-28T11:52:55.014 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.012+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bd809b6b0 con 0x7f3bb80682f0 2023-11-28T11:52:55.021 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.020+0000 7f3bc6ffd700 1 -- 172.21.15.144:0/338163937 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302768 (secure 0 0 0) 0x7f3bcc022070 con 0x7f3bb80682f0 2023-11-28T11:52:55.025 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:55.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 msgr2=0x7f3bb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8071070 tx=0x7f3bcc007000 comp rx=0 tx=0).stop 2023-11-28T11:52:55.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 msgr2=0x7f3bd81efb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bd81efb40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8011aa0 tx=0x7f3bc8011e60 comp rx=0 tx=0).stop 2023-11-28T11:52:55.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 shutdown_connections 2023-11-28T11:52:55.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bc0004430 0x7f3bd81efb40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3bb80682f0 0x7f3bb806a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bd8071ac0 0x7f3bd80c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 --2- 172.21.15.144:0/338163937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3bd814ec40 0x7f3bd8071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 >> 172.21.15.144:0/338163937 conn(0x7f3bd80d0000 msgr2=0x7f3bd806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:55.029 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 shutdown_connections 2023-11-28T11:52:55.030 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.023+0000 7f3bdc9d4700 1 -- 172.21.15.144:0/338163937 wait complete. 2023-11-28T11:52:55.030 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:55.037 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-28T11:52:54.503286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71988,"kb_used_data":5044,"kb_used_omap":0,"kb_used_meta":66880,"kb_avail":754902732,"statfs":{"total":773094113280,"available":773020397568,"internally_reserved":0,"allocated":5165056,"data_stored":2835378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68485120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.695937"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.823962+0000","last_change":"2023-11-28T11:52:49.823962+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.823962+0000","last_undegraded":"2023-11-28T11:52:49.823962+0000","last_fullsized":"2023-11-28T11:52:49.823962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:11.475783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.824712+0000","last_change":"2023-11-28T11:52:49.824712+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.824712+0000","last_undegraded":"2023-11-28T11:52:49.824712+0000","last_fullsized":"2023-11-28T11:52:49.824712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:37.366923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822634+0000","last_change":"2023-11-28T11:52:49.822634+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822634+0000","last_undegraded":"2023-11-28T11:52:49.822634+0000","last_fullsized":"2023-11-28T11:52:49.822634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:28.225138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822447+0000","last_change":"2023-11-28T11:52:49.822447+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822447+0000","last_undegraded":"2023-11-28T11:52:49.822447+0000","last_fullsized":"2023-11-28T11:52:49.822447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.465684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822496+0000","last_change":"2023-11-28T11:52:49.822496+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822496+0000","last_undegraded":"2023-11-28T11:52:49.822496+0000","last_fullsized":"2023-11-28T11:52:49.822496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:54:06.429788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817298+0000","last_change":"2023-11-28T11:52:49.817298+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817298+0000","last_undegraded":"2023-11-28T11:52:49.817298+0000","last_fullsized":"2023-11-28T11:52:49.817298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:53.755885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817657+0000","last_change":"2023-11-28T11:52:49.817657+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817657+0000","last_undegraded":"2023-11-28T11:52:49.817657+0000","last_fullsized":"2023-11-28T11:52:49.817657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:52.523879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815808+0000","last_change":"2023-11-28T11:52:49.815808+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815808+0000","last_undegraded":"2023-11-28T11:52:49.815808+0000","last_fullsized":"2023-11-28T11:52:49.815808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:00.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815142+0000","last_change":"2023-11-28T11:52:49.815142+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815142+0000","last_undegraded":"2023-11-28T11:52:49.815142+0000","last_fullsized":"2023-11-28T11:52:49.815142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:42.026408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.809482+0000","last_change":"2023-11-28T11:52:48.809482+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.809482+0000","last_undegraded":"2023-11-28T11:52:48.809482+0000","last_fullsized":"2023-11-28T11:52:48.809482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:54.890718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813849+0000","last_change":"2023-11-28T11:52:48.813849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813849+0000","last_undegraded":"2023-11-28T11:52:48.813849+0000","last_fullsized":"2023-11-28T11:52:48.813849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813794+0000","last_change":"2023-11-28T11:52:48.813794+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813794+0000","last_undegraded":"2023-11-28T11:52:48.813794+0000","last_fullsized":"2023-11-28T11:52:48.813794+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815867+0000","last_change":"2023-11-28T11:52:48.815867+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815867+0000","last_undegraded":"2023-11-28T11:52:48.815867+0000","last_fullsized":"2023-11-28T11:52:48.815867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:02.973674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829812+0000","last_change":"2023-11-28T11:52:49.827097+0000","last_active":"2023-11-28T11:52:49.829812+0000","last_peered":"2023-11-28T11:52:49.829812+0000","last_clean":"2023-11-28T11:52:49.829812+0000","last_became_active":"2023-11-28T11:52:49.826826+0000","last_became_peered":"2023-11-28T11:52:49.826826+0000","last_unstale":"2023-11-28T11:52:49.829812+0000","last_undegraded":"2023-11-28T11:52:49.829812+0000","last_fullsized":"2023-11-28T11:52:49.829812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:23.759602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811534+0000","last_change":"2023-11-28T11:52:48.811534+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811534+0000","last_undegraded":"2023-11-28T11:52:48.811534+0000","last_fullsized":"2023-11-28T11:52:48.811534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808962+0000","last_change":"2023-11-28T11:52:48.808962+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808962+0000","last_undegraded":"2023-11-28T11:52:48.808962+0000","last_fullsized":"2023-11-28T11:52:48.808962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:22.861924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829014+0000","last_change":"2023-11-28T11:52:49.822322+0000","last_active":"2023-11-28T11:52:49.829014+0000","last_peered":"2023-11-28T11:52:49.829014+0000","last_clean":"2023-11-28T11:52:49.829014+0000","last_became_active":"2023-11-28T11:52:49.822077+0000","last_became_peered":"2023-11-28T11:52:49.822077+0000","last_unstale":"2023-11-28T11:52:49.829014+0000","last_undegraded":"2023-11-28T11:52:49.829014+0000","last_fullsized":"2023-11-28T11:52:49.829014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:24.019745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814558+0000","last_change":"2023-11-28T11:52:48.814558+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814558+0000","last_undegraded":"2023-11-28T11:52:48.814558+0000","last_fullsized":"2023-11-28T11:52:48.814558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:50.368503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.812849+0000","last_change":"2023-11-28T11:52:48.812849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.812849+0000","last_undegraded":"2023-11-28T11:52:48.812849+0000","last_fullsized":"2023-11-28T11:52:48.812849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:22.526251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814234+0000","last_change":"2023-11-28T11:52:49.814234+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814234+0000","last_undegraded":"2023-11-28T11:52:49.814234+0000","last_fullsized":"2023-11-28T11:52:49.814234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:30.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814475+0000","last_change":"2023-11-28T11:52:48.814475+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814475+0000","last_undegraded":"2023-11-28T11:52:48.814475+0000","last_fullsized":"2023-11-28T11:52:48.814475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:46.875258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810166+0000","last_change":"2023-11-28T11:52:48.810166+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810166+0000","last_undegraded":"2023-11-28T11:52:48.810166+0000","last_fullsized":"2023-11-28T11:52:48.810166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:16.135396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834400+0000","last_change":"2023-11-28T11:52:49.830260+0000","last_active":"2023-11-28T11:52:49.834400+0000","last_peered":"2023-11-28T11:52:49.834400+0000","last_clean":"2023-11-28T11:52:49.834400+0000","last_became_active":"2023-11-28T11:52:49.829977+0000","last_became_peered":"2023-11-28T11:52:49.829977+0000","last_unstale":"2023-11-28T11:52:49.834400+0000","last_undegraded":"2023-11-28T11:52:49.834400+0000","last_fullsized":"2023-11-28T11:52:49.834400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:19.733169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814190+0000","last_change":"2023-11-28T11:52:49.814190+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814190+0000","last_undegraded":"2023-11-28T11:52:49.814190+0000","last_fullsized":"2023-11-28T11:52:49.814190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:03.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813836+0000","last_change":"2023-11-28T11:52:48.813836+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813836+0000","last_undegraded":"2023-11-28T11:52:48.813836+0000","last_fullsized":"2023-11-28T11:52:48.813836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.978385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.816827+0000","last_change":"2023-11-28T11:52:49.816827+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.816827+0000","last_undegraded":"2023-11-28T11:52:49.816827+0000","last_fullsized":"2023-11-28T11:52:49.816827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:30.533622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:44.118206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829400+0000","last_change":"2023-11-28T11:52:49.826554+0000","last_active":"2023-11-28T11:52:49.829400+0000","last_peered":"2023-11-28T11:52:49.829400+0000","last_clean":"2023-11-28T11:52:49.829400+0000","last_became_active":"2023-11-28T11:52:49.826299+0000","last_became_peered":"2023-11-28T11:52:49.826299+0000","last_unstale":"2023-11-28T11:52:49.829400+0000","last_undegraded":"2023-11-28T11:52:49.829400+0000","last_fullsized":"2023-11-28T11:52:49.829400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:33.111523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813679+0000","last_change":"2023-11-28T11:52:48.813679+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813679+0000","last_undegraded":"2023-11-28T11:52:48.813679+0000","last_fullsized":"2023-11-28T11:52:48.813679+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:51.984789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808979+0000","last_change":"2023-11-28T11:52:48.808979+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808979+0000","last_undegraded":"2023-11-28T11:52:48.808979+0000","last_fullsized":"2023-11-28T11:52:48.808979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:23.803406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813777+0000","last_change":"2023-11-28T11:52:48.813777+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813777+0000","last_undegraded":"2023-11-28T11:52:48.813777+0000","last_fullsized":"2023-11-28T11:52:48.813777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810666+0000","last_change":"2023-11-28T11:52:48.810666+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810666+0000","last_undegraded":"2023-11-28T11:52:48.810666+0000","last_fullsized":"2023-11-28T11:52:48.810666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:54.122734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814746+0000","last_change":"2023-11-28T11:52:49.814746+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814746+0000","last_undegraded":"2023-11-28T11:52:49.814746+0000","last_fullsized":"2023-11-28T11:52:49.814746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:26.868306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834375+0000","last_change":"2023-11-28T11:52:49.829671+0000","last_active":"2023-11-28T11:52:49.834375+0000","last_peered":"2023-11-28T11:52:49.834375+0000","last_clean":"2023-11-28T11:52:49.834375+0000","last_became_active":"2023-11-28T11:52:49.829404+0000","last_became_peered":"2023-11-28T11:52:49.829404+0000","last_unstale":"2023-11-28T11:52:49.834375+0000","last_undegraded":"2023-11-28T11:52:49.834375+0000","last_fullsized":"2023-11-28T11:52:49.834375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:18.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813638+0000","last_change":"2023-11-28T11:52:48.813638+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813638+0000","last_undegraded":"2023-11-28T11:52:48.813638+0000","last_fullsized":"2023-11-28T11:52:48.813638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:57.828731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814357+0000","last_change":"2023-11-28T11:52:49.814357+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814357+0000","last_undegraded":"2023-11-28T11:52:49.814357+0000","last_fullsized":"2023-11-28T11:52:49.814357+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:04.510511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810139+0000","last_change":"2023-11-28T11:52:48.810139+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810139+0000","last_undegraded":"2023-11-28T11:52:48.810139+0000","last_fullsized":"2023-11-28T11:52:48.810139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:39.463589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286150+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:46.286150+0000","last_peered":"2023-11-28T11:52:46.286150+0000","last_clean":"2023-11-28T11:52:46.286150+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:46.286150+0000","last_undegraded":"2023-11-28T11:52:46.286150+0000","last_fullsized":"2023-11-28T11:52:46.286150+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:33.980547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132093+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:46.132093+0000","last_peered":"2023-11-28T11:52:46.132093+0000","last_clean":"2023-11-28T11:52:46.132093+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:46.132093+0000","last_undegraded":"2023-11-28T11:52:46.132093+0000","last_fullsized":"2023-11-28T11:52:46.132093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813768+0000","last_change":"2023-11-28T11:52:48.813768+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813768+0000","last_undegraded":"2023-11-28T11:52:48.813768+0000","last_fullsized":"2023-11-28T11:52:48.813768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:34.991301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.285986+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:46.285986+0000","last_peered":"2023-11-28T11:52:46.285986+0000","last_clean":"2023-11-28T11:52:46.285986+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:46.285986+0000","last_undegraded":"2023-11-28T11:52:46.285986+0000","last_fullsized":"2023-11-28T11:52:46.285986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":181,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132872+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:46.132872+0000","last_peered":"2023-11-28T11:52:46.132872+0000","last_clean":"2023-11-28T11:52:46.132872+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:46.132872+0000","last_undegraded":"2023-11-28T11:52:46.132872+0000","last_fullsized":"2023-11-28T11:52:46.132872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829752+0000","last_change":"2023-11-28T11:52:49.824383+0000","last_active":"2023-11-28T11:52:49.829752+0000","last_peered":"2023-11-28T11:52:49.829752+0000","last_clean":"2023-11-28T11:52:49.829752+0000","last_became_active":"2023-11-28T11:52:49.824081+0000","last_became_peered":"2023-11-28T11:52:49.824081+0000","last_unstale":"2023-11-28T11:52:49.829752+0000","last_undegraded":"2023-11-28T11:52:49.829752+0000","last_fullsized":"2023-11-28T11:52:49.829752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:17.464586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132870+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:46.132870+0000","last_peered":"2023-11-28T11:52:46.132870+0000","last_clean":"2023-11-28T11:52:46.132870+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:46.132870+0000","last_undegraded":"2023-11-28T11:52:46.132870+0000","last_fullsized":"2023-11-28T11:52:46.132870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834419+0000","last_change":"2023-11-28T11:52:49.830269+0000","last_active":"2023-11-28T11:52:49.834419+0000","last_peered":"2023-11-28T11:52:49.834419+0000","last_clean":"2023-11-28T11:52:49.834419+0000","last_became_active":"2023-11-28T11:52:49.830011+0000","last_became_peered":"2023-11-28T11:52:49.830011+0000","last_unstale":"2023-11-28T11:52:49.834419+0000","last_undegraded":"2023-11-28T11:52:49.834419+0000","last_fullsized":"2023-11-28T11:52:49.834419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:05.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132859+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:46.132859+0000","last_peered":"2023-11-28T11:52:46.132859+0000","last_clean":"2023-11-28T11:52:46.132859+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:46.132859+0000","last_undegraded":"2023-11-28T11:52:46.132859+0000","last_fullsized":"2023-11-28T11:52:46.132859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808847+0000","last_change":"2023-11-28T11:52:48.808847+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808847+0000","last_undegraded":"2023-11-28T11:52:48.808847+0000","last_fullsized":"2023-11-28T11:52:48.808847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286133+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:46.286133+0000","last_peered":"2023-11-28T11:52:46.286133+0000","last_clean":"2023-11-28T11:52:46.286133+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:46.286133+0000","last_undegraded":"2023-11-28T11:52:46.286133+0000","last_fullsized":"2023-11-28T11:52:46.286133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815863+0000","last_change":"2023-11-28T11:52:48.815863+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815863+0000","last_undegraded":"2023-11-28T11:52:48.815863+0000","last_fullsized":"2023-11-28T11:52:48.815863+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131458+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:46.131458+0000","last_peered":"2023-11-28T11:52:46.131458+0000","last_clean":"2023-11-28T11:52:46.131458+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:46.131458+0000","last_undegraded":"2023-11-28T11:52:46.131458+0000","last_fullsized":"2023-11-28T11:52:46.131458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815176+0000","last_change":"2023-11-28T11:52:49.815176+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815176+0000","last_undegraded":"2023-11-28T11:52:49.815176+0000","last_fullsized":"2023-11-28T11:52:49.815176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:55.616287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811601+0000","last_change":"2023-11-28T11:52:48.811601+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811601+0000","last_undegraded":"2023-11-28T11:52:48.811601+0000","last_fullsized":"2023-11-28T11:52:48.811601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:48:34.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131493+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:46.131493+0000","last_peered":"2023-11-28T11:52:46.131493+0000","last_clean":"2023-11-28T11:52:46.131493+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:46.131493+0000","last_undegraded":"2023-11-28T11:52:46.131493+0000","last_fullsized":"2023-11-28T11:52:46.131493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9896,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361944,"statfs":{"total":96636764160,"available":96626630656,"internally_reserved":0,"allocated":1081344,"data_stored":792046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59899999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":3.7810000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9716,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362124,"statfs":{"total":96636764160,"available":96626814976,"internally_reserved":0,"allocated":1093632,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.30199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75700000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8684,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363156,"statfs":{"total":96636764160,"available":96627871744,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.32900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.317}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.501}]}]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8880,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362960,"statfs":{"total":96636764160,"available":96627671040,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.254},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.30199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":16.231000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.755},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.441},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:55.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.823962+0000', 'last_change': '2023-11-28T11:52:49.823962+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.823962+0000', 'last_undegraded': '2023-11-28T11:52:49.823962+0000', 'last_fullsized': '2023-11-28T11:52:49.823962+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:54:11.475783+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.824712+0000', 'last_change': '2023-11-28T11:52:49.824712+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.824712+0000', 'last_undegraded': '2023-11-28T11:52:49.824712+0000', 'last_fullsized': '2023-11-28T11:52:49.824712+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:57:37.366923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.822634+0000', 'last_change': '2023-11-28T11:52:49.822634+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.822634+0000', 'last_undegraded': '2023-11-28T11:52:49.822634+0000', 'last_fullsized': '2023-11-28T11:52:49.822634+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:51:28.225138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2023-11-28T11:52:55.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.822447+0000', 'last_change': '2023-11-28T11:52:49.822447+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.822447+0000', 'last_undegraded': '2023-11-28T11:52:49.822447+0000', 'last_fullsized': '2023-11-28T11:52:49.822447+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:09:26.465684+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.822496+0000', 'last_change': '2023-11-28T11:52:49.822496+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.822496+0000', 'last_undegraded': '2023-11-28T11:52:49.822496+0000', 'last_fullsized': '2023-11-28T11:52:49.822496+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T11:54:06.429788+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.817298+0000', 'last_change': '2023-11-28T11:52:49.817298+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.817298+0000', 'last_undegraded': '2023-11-28T11:52:49.817298+0000', 'last_fullsized': '2023-11-28T11:52:49.817298+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:19:53.755885+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.817657+0000', 'last_change': '2023-11-28T11:52:49.817657+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.817657+0000', 'last_undegraded': '2023-11-28T11:52:49.817657+0000', 'last_fullsized': '2023-11-28T11:52:49.817657+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:01:52.523879+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2023-11-28T11:52:55.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.815808+0000', 'last_change': '2023-11-28T11:52:49.815808+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.815808+0000', 'last_undegraded': '2023-11-28T11:52:49.815808+0000', 'last_fullsized': '2023-11-28T11:52:49.815808+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:10:00.114504+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2023-11-28T11:52:55.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.815142+0000', 'last_change': '2023-11-28T11:52:49.815142+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.815142+0000', 'last_undegraded': '2023-11-28T11:52:49.815142+0000', 'last_fullsized': '2023-11-28T11:52:49.815142+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:20:42.026408+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.809482+0000', 'last_change': '2023-11-28T11:52:48.809482+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.809482+0000', 'last_undegraded': '2023-11-28T11:52:48.809482+0000', 'last_fullsized': '2023-11-28T11:52:48.809482+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:06:54.890718+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813849+0000', 'last_change': '2023-11-28T11:52:48.813849+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813849+0000', 'last_undegraded': '2023-11-28T11:52:48.813849+0000', 'last_fullsized': '2023-11-28T11:52:48.813849+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:52:55.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813794+0000', 'last_change': '2023-11-28T11:52:48.813794+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813794+0000', 'last_undegraded': '2023-11-28T11:52:48.813794+0000', 'last_fullsized': '2023-11-28T11:52:48.813794+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.815867+0000', 'last_change': '2023-11-28T11:52:48.815867+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.815867+0000', 'last_undegraded': '2023-11-28T11:52:48.815867+0000', 'last_fullsized': '2023-11-28T11:52:48.815867+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:38:02.973674+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:52:55.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.811534+0000', 'last_change': '2023-11-28T11:52:48.811534+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.811534+0000', 'last_undegraded': '2023-11-28T11:52:48.811534+0000', 'last_fullsized': '2023-11-28T11:52:48.811534+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.808962+0000', 'last_change': '2023-11-28T11:52:48.808962+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.808962+0000', 'last_undegraded': '2023-11-28T11:52:48.808962+0000', 'last_fullsized': '2023-11-28T11:52:48.808962+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:28:22.861924+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-11-28T11:52:55.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.814558+0000', 'last_change': '2023-11-28T11:52:48.814558+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.814558+0000', 'last_undegraded': '2023-11-28T11:52:48.814558+0000', 'last_fullsized': '2023-11-28T11:52:48.814558+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:23:50.368503+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.812849+0000', 'last_change': '2023-11-28T11:52:48.812849+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.812849+0000', 'last_undegraded': '2023-11-28T11:52:48.812849+0000', 'last_fullsized': '2023-11-28T11:52:48.812849+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:25:22.526251+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-11-28T11:52:55.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.814234+0000', 'last_change': '2023-11-28T11:52:49.814234+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.814234+0000', 'last_undegraded': '2023-11-28T11:52:49.814234+0000', 'last_fullsized': '2023-11-28T11:52:49.814234+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:00:30.900577+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.814475+0000', 'last_change': '2023-11-28T11:52:48.814475+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.814475+0000', 'last_undegraded': '2023-11-28T11:52:48.814475+0000', 'last_fullsized': '2023-11-28T11:52:48.814475+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:50:46.875258+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.810166+0000', 'last_change': '2023-11-28T11:52:48.810166+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.810166+0000', 'last_undegraded': '2023-11-28T11:52:48.810166+0000', 'last_fullsized': '2023-11-28T11:52:48.810166+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:50:16.135396+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.814190+0000', 'last_change': '2023-11-28T11:52:49.814190+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.814190+0000', 'last_undegraded': '2023-11-28T11:52:49.814190+0000', 'last_fullsized': '2023-11-28T11:52:49.814190+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:10:03.188686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.808845+0000', 'last_change': '2023-11-28T11:52:48.808845+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.808845+0000', 'last_undegraded': '2023-11-28T11:52:48.808845+0000', 'last_fullsized': '2023-11-28T11:52:48.808845+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813836+0000', 'last_change': '2023-11-28T11:52:48.813836+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813836+0000', 'last_undegraded': '2023-11-28T11:52:48.813836+0000', 'last_fullsized': '2023-11-28T11:52:48.813836+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:20:15.978385+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.816827+0000', 'last_change': '2023-11-28T11:52:49.816827+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.816827+0000', 'last_undegraded': '2023-11-28T11:52:49.816827+0000', 'last_fullsized': '2023-11-28T11:52:49.816827+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:57:30.533622+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.810136+0000', 'last_change': '2023-11-28T11:52:48.810136+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.810136+0000', 'last_undegraded': '2023-11-28T11:52:48.810136+0000', 'last_fullsized': '2023-11-28T11:52:48.810136+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:37:44.118206+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:52:55.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813679+0000', 'last_change': '2023-11-28T11:52:48.813679+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813679+0000', 'last_undegraded': '2023-11-28T11:52:48.813679+0000', 'last_fullsized': '2023-11-28T11:52:48.813679+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:24:51.984789+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.808979+0000', 'last_change': '2023-11-28T11:52:48.808979+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.808979+0000', 'last_undegraded': '2023-11-28T11:52:48.808979+0000', 'last_fullsized': '2023-11-28T11:52:48.808979+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:52:55.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.808845+0000', 'last_change': '2023-11-28T11:52:48.808845+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.808845+0000', 'last_undegraded': '2023-11-28T11:52:48.808845+0000', 'last_fullsized': '2023-11-28T11:52:48.808845+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:10:23.803406+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813777+0000', 'last_change': '2023-11-28T11:52:48.813777+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813777+0000', 'last_undegraded': '2023-11-28T11:52:48.813777+0000', 'last_fullsized': '2023-11-28T11:52:48.813777+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-28T11:52:55.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.810666+0000', 'last_change': '2023-11-28T11:52:48.810666+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.810666+0000', 'last_undegraded': '2023-11-28T11:52:48.810666+0000', 'last_fullsized': '2023-11-28T11:52:48.810666+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:13:54.122734+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.814746+0000', 'last_change': '2023-11-28T11:52:49.814746+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.814746+0000', 'last_undegraded': '2023-11-28T11:52:49.814746+0000', 'last_fullsized': '2023-11-28T11:52:49.814746+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:55:26.868306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813638+0000', 'last_change': '2023-11-28T11:52:48.813638+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813638+0000', 'last_undegraded': '2023-11-28T11:52:48.813638+0000', 'last_fullsized': '2023-11-28T11:52:48.813638+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T22:11:57.828731+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.814357+0000', 'last_change': '2023-11-28T11:52:49.814357+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.814357+0000', 'last_undegraded': '2023-11-28T11:52:49.814357+0000', 'last_fullsized': '2023-11-28T11:52:49.814357+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:49:04.510511+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-28T11:52:55.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.810139+0000', 'last_change': '2023-11-28T11:52:48.810139+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.810139+0000', 'last_undegraded': '2023-11-28T11:52:48.810139+0000', 'last_fullsized': '2023-11-28T11:52:48.810139+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:10:39.463589+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.810136+0000', 'last_change': '2023-11-28T11:52:48.810136+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.810136+0000', 'last_undegraded': '2023-11-28T11:52:48.810136+0000', 'last_fullsized': '2023-11-28T11:52:48.810136+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:04:33.980547+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.813768+0000', 'last_change': '2023-11-28T11:52:48.813768+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.813768+0000', 'last_undegraded': '2023-11-28T11:52:48.813768+0000', 'last_fullsized': '2023-11-28T11:52:48.813768+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:53:34.991301+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-11-28T11:52:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.808847+0000', 'last_change': '2023-11-28T11:52:48.808847+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.808847+0000', 'last_undegraded': '2023-11-28T11:52:48.808847+0000', 'last_fullsized': '2023-11-28T11:52:48.808847+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:52:49.804716+0000', 'last_change': '2023-11-28T11:52:49.804716+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.804716+0000', 'last_undegraded': '2023-11-28T11:52:49.804716+0000', 'last_fullsized': '2023-11-28T11:52:49.804716+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.815863+0000', 'last_change': '2023-11-28T11:52:48.815863+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.815863+0000', 'last_undegraded': '2023-11-28T11:52:48.815863+0000', 'last_fullsized': '2023-11-28T11:52:48.815863+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:52:55.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-11-28T11:52:55.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:49.815176+0000', 'last_change': '2023-11-28T11:52:49.815176+0000', 'last_active': '2023-11-28T11:52:49.804716+0000', 'last_peered': '2023-11-28T11:52:49.804716+0000', 'last_clean': '2023-11-28T11:52:49.804716+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:49.815176+0000', 'last_undegraded': '2023-11-28T11:52:49.815176+0000', 'last_fullsized': '2023-11-28T11:52:49.815176+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:49.804716+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:27:55.616287+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:52:55.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-28T11:52:55.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:52:48.811601+0000', 'last_change': '2023-11-28T11:52:48.811601+0000', 'last_active': '2023-11-28T11:52:48.796034+0000', 'last_peered': '2023-11-28T11:52:48.796034+0000', 'last_clean': '2023-11-28T11:52:48.796034+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:52:48.811601+0000', 'last_undegraded': '2023-11-28T11:52:48.811601+0000', 'last_fullsized': '2023-11-28T11:52:48.811601+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'last_clean_scrub_stamp': '2023-11-28T11:52:48.796034+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:48:34.060703+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:52:55.060 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T11:52:55.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.195+0000 7f2d468f4700 1 Processor -- start 2023-11-28T11:52:55.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d468f4700 1 -- start start 2023-11-28T11:52:55.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d468f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 0x7f2d400af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d468f4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400cc5f0 con 0x7f2d4015cef0 2023-11-28T11:52:55.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d468f4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4015d470 con 0x7f2d4006ba90 2023-11-28T11:52:55.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d468f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015d750 con 0x7f2d400cbdf0 2023-11-28T11:52:55.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d3ffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f2d4015cef0 0x7f2d4006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54142/0 (socket says 172.21.15.144:54142) 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.196+0000 7f2d3ffff700 1 -- 172.21.15.144:0/3062603396 learned_addr learned my addr 172.21.15.144:0/3062603396 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d44e91700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 0x7f2d400af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570498320 0 0) 0x7f2d400cc5f0 con 0x7f2d4015cef0 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d280035b0 con 0x7f2d4015cef0 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1594593946 0 0) 0x7f2d280035b0 con 0x7f2d4015cef0 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 msgr2=0x7f2d400af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 0x7f2d400af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 >> v1:172.21.15.177:6789/0 conn(0x7f2d4006ba90 legacy=0x7f2d400ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4015da30 con 0x7f2d4015cef0 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f2d30001670 con 0x7f2d4015cef0 2023-11-28T11:52:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 >> v1:172.21.15.144:6789/0 conn(0x7f2d4015cef0 legacy=0x7f2d4006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d28003a30 0x7f2d28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d28004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 0x7f2d280050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015d750 con 0x7f2d280043d0 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2800ba10 con 0x7f2d28003a30 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2800bba0 con 0x7f2d400cbdf0 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d28004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3ffff700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d28003a30 0x7f2d28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d44e91700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 0x7f2d280050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3f7fe700 1 -- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 msgr2=0x7f2d280050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400cbdf0 0x7f2d280050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3f7fe700 1 -- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d28003a30 msgr2=0x7f2d28003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d28003a30 0x7f2d28003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.197+0000 7f2d3f7fe700 1 -- 172.21.15.144:0/3062603396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d280035b0 con 0x7f2d280043d0 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d28004890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2d340018d0 tx=0x7f2d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a53a6c5aa8cf14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400c690 con 0x7f2d280043d0 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3400c7f0 con 0x7f2d280043d0 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d3effd700 1 -- 172.21.15.144:0/3062603396 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400cba0 con 0x7f2d280043d0 2023-11-28T11:52:55.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d468f4700 1 -- 172.21.15.144:0/3062603396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 msgr2=0x7f2d28004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d468f4700 1 --2- 172.21.15.144:0/3062603396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d28004890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2d340018d0 tx=0x7f2d28005880 comp rx=0 tx=0).stop 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d44e91700 1 -- 172.21.15.144:0/3062603396 reap_dead start 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d468f4700 1 -- 172.21.15.144:0/3062603396 shutdown_connections 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d468f4700 1 -- 172.21.15.144:0/3062603396 >> 172.21.15.144:0/3062603396 conn(0x7f2d400d0000 msgr2=0x7f2d400d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d468f4700 1 -- 172.21.15.144:0/3062603396 shutdown_connections 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.198+0000 7f2d468f4700 1 -- 172.21.15.144:0/3062603396 wait complete. 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.199+0000 7f2d468f4700 1 Processor -- start 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 -- start start 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d400ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400b5860 0x7f2d400c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d401f2040 0x7f2d400b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401591e0 con 0x7f2d280043d0 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400c48c0 con 0x7f2d401f2040 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.200+0000 7f2d468f4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400c3c40 con 0x7f2d400b5860 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d400ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d400ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37442/0 (socket says 172.21.15.144:37442) 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 -- 172.21.15.144:0/502789746 learned_addr learned my addr 172.21.15.144:0/502789746 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400b5860 0x7f2d400c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d44e91700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d401f2040 0x7f2d400b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 -- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400b5860 msgr2=0x7f2d400c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400b5860 0x7f2d400c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 -- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d401f2040 msgr2=0x7f2d400b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d401f2040 0x7f2d400b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 -- 172.21.15.144:0/502789746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4015dff0 con 0x7f2d280043d0 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3ffff700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d400ae480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2d300072c0 tx=0x7f2d30007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300077a0 con 0x7f2d280043d0 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3000f850 con 0x7f2d280043d0 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4015dd10 con 0x7f2d280043d0 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000fbe0 con 0x7f2d280043d0 2023-11-28T11:52:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.201+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d400af400 con 0x7f2d280043d0 2023-11-28T11:52:55.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.202+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d400d0e60 con 0x7f2d280043d0 2023-11-28T11:52:55.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.203+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d3000fd40 con 0x7f2d280043d0 2023-11-28T11:52:55.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.203+0000 7f2d3cff9700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d2c0682f0 0x7f2d2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.203+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d30085db0 con 0x7f2d280043d0 2023-11-28T11:52:55.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.203+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d2c0682f0 0x7f2d2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.203+0000 7f2d3f7fe700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d2c0682f0 0x7f2d2c06a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d401f31b0 tx=0x7f2d34005040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:55.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.207+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d300526c0 con 0x7f2d280043d0 2023-11-28T11:52:55.468 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.467+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2d4015c7d0 con 0x7f2d280043d0 2023-11-28T11:52:55.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.467+0000 7f2d3cff9700 1 -- 172.21.15.144:0/502789746 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1655 (secure 0 0 0) 0x7f2d30055ce0 con 0x7f2d280043d0 2023-11-28T11:52:55.469 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:55.469 INFO:teuthology.orchestra.run.smithi144.stdout:{"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":116,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1701172269,"num_in_osds":8,"osd_in_since":1701172261,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":49},{"state_name":"creating+peering","count":40},{"state_name":"active+clean","count":16}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":73715712,"bytes_avail":773020397568,"bytes_total":773094113280,"unknown_pgs_ratio":0.46666666865348816,"inactive_pgs_ratio":0.380952388048172},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5535}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-11-28T11:52:10.494121+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d2c0682f0 msgr2=0x7f2d2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d2c0682f0 0x7f2d2c06a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d401f31b0 tx=0x7f2d34005040 comp rx=0 tx=0).stop 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 msgr2=0x7f2d400ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d400ae480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2d300072c0 tx=0x7f2d30007680 comp rx=0 tx=0).stop 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 shutdown_connections 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d280043d0 0x7f2d400ae480 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d2c0682f0 0x7f2d2c06a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d401f2040 0x7f2d400b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 --2- 172.21.15.144:0/502789746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d400b5860 0x7f2d400c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 >> 172.21.15.144:0/502789746 conn(0x7f2d400d0000 msgr2=0x7f2d400cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:55.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 shutdown_connections 2023-11-28T11:52:55.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.469+0000 7f2d468f4700 1 -- 172.21.15.144:0/502789746 wait complete. 2023-11-28T11:52:55.483 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T11:52:55.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:55.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.602+0000 7f460abeb700 1 Processor -- start 2023-11-28T11:52:55.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.602+0000 7f460abeb700 1 -- start start 2023-11-28T11:52:55.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f460abeb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f460415cef0 0x7f46040af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f460abeb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 0x7f46040ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f460abeb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46040cc5f0 con 0x7f46040cbdf0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f460abeb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460415d470 con 0x7f460415cef0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f460abeb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460415d750 con 0x7f460406ba90 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f4603fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f46040cbdf0 0x7f460406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54164/0 (socket says 172.21.15.144:54164) 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f4603fff700 1 -- 172.21.15.144:0/3937432484 learned_addr learned my addr 172.21.15.144:0/3937432484 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f4609188700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 0x7f46040ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f4608987700 1 --2- 172.21.15.144:0/3937432484 >> v2:172.21.15.177:3300/0 conn(0x7f460415cef0 0x7f46040af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385126085 0 0) 0x7f46040cc5f0 con 0x7f46040cbdf0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f40035b0 con 0x7f46040cbdf0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260682285 0 0) 0x7f45f40035b0 con 0x7f46040cbdf0 2023-11-28T11:52:55.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 msgr2=0x7f46040ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 0x7f46040ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 >> v2:172.21.15.177:3300/0 conn(0x7f460415cef0 msgr2=0x7f46040af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 --2- 172.21.15.144:0/3937432484 >> v2:172.21.15.177:3300/0 conn(0x7f460415cef0 0x7f46040af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460415da30 con 0x7f46040cbdf0 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f45f0001670 con 0x7f46040cbdf0 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 >> v1:172.21.15.144:6789/0 conn(0x7f46040cbdf0 legacy=0x7f460406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 0x7f45f4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.603+0000 7f46037fe700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f4003f80 0x7f45f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f4004d90 0x7f45f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460415d470 con 0x7f45f4004d90 2023-11-28T11:52:55.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460415d750 con 0x7f45f4003f80 2023-11-28T11:52:55.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f400ba40 con 0x7f460406ba90 2023-11-28T11:52:55.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f4004d90 0x7f45f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4608987700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 0x7f45f4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4603fff700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f4003f80 0x7f45f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 -- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 msgr2=0x7f45f4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f460406ba90 0x7f45f4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 -- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f4003f80 msgr2=0x7f45f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45f4003f80 0x7f45f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 -- 172.21.15.144:0/3937432484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40035b0 con 0x7f45f4004d90 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f4609188700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f4004d90 0x7f45f4005200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45fc010450 tx=0x7f45f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14cb6bdc1b3741a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc010e90 con 0x7f45f4004d90 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc012430 con 0x7f45f4004d90 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.604+0000 7f46037fe700 1 -- 172.21.15.144:0/3937432484 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0127c0 con 0x7f45f4004d90 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f460abeb700 1 -- 172.21.15.144:0/3937432484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f4004d90 msgr2=0x7f45f4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f460abeb700 1 --2- 172.21.15.144:0/3937432484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f4004d90 0x7f45f4005200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45fc010450 tx=0x7f45f4005880 comp rx=0 tx=0).stop 2023-11-28T11:52:55.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f4609188700 1 -- 172.21.15.144:0/3937432484 reap_dead start 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f460abeb700 1 -- 172.21.15.144:0/3937432484 shutdown_connections 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f460abeb700 1 -- 172.21.15.144:0/3937432484 >> 172.21.15.144:0/3937432484 conn(0x7f46040d0000 msgr2=0x7f46040d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f460abeb700 1 -- 172.21.15.144:0/3937432484 shutdown_connections 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.605+0000 7f460abeb700 1 -- 172.21.15.144:0/3937432484 wait complete. 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.606+0000 7f460abeb700 1 Processor -- start 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.606+0000 7f460abeb700 1 -- start start 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f460abeb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 0x7f46041f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f460abeb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4604157d20 0x7f46040c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f460abeb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46040c5830 0x7f46040ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f460abeb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040c3f10 con 0x7f46040c6420 2023-11-28T11:52:55.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f460abeb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604156260 con 0x7f46040c5830 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f460abeb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46041f6730 con 0x7f4604157d20 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 0x7f46041f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 0x7f46041f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37462/0 (socket says 172.21.15.144:37462) 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 -- 172.21.15.144:0/576050743 learned_addr learned my addr 172.21.15.144:0/576050743 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4609188700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46040c5830 0x7f46040ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4603fff700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4604157d20 0x7f46040c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 -- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4604157d20 msgr2=0x7f46040c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4604157d20 0x7f46040c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 -- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46040c5830 msgr2=0x7f46040ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46040c5830 0x7f46040ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 -- 172.21.15.144:0/576050743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460415dff0 con 0x7f46040c6420 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.607+0000 7f4608987700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 0x7f46041f5ee0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f45f8011aa0 tx=0x7f45f8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.608+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800f9e0 con 0x7f46040c6420 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.608+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800fb40 con 0x7f46040c6420 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.608+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460415dd10 con 0x7f46040c6420 2023-11-28T11:52:55.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.608+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8014440 con 0x7f46040c6420 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.608+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460406a270 con 0x7f46040c6420 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.609+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604156870 con 0x7f46040c6420 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.609+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45f80145a0 con 0x7f46040c6420 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.609+0000 7f46017fa700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45e80682f0 0x7f45e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.610+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45f8086660 con 0x7f46040c6420 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.610+0000 7f4603fff700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45e80682f0 0x7f45e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:55.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.610+0000 7f4603fff700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45e80682f0 0x7f45e806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f45f00048d0 tx=0x7f45f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:55.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.614+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f45f8052e60 con 0x7f46040c6420 2023-11-28T11:52:55.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.791+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f460406e990 con 0x7f45e80682f0 2023-11-28T11:52:55.799 INFO:tasks.ceph.mds.d.smithi177.stderr:2023-11-28T11:52:55.798+0000 7f318610b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:52:55.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.799+0000 7f46017fa700 1 -- 172.21.15.144:0/576050743 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302768 (secure 0 0 0) 0x7f460406e990 con 0x7f45e80682f0 2023-11-28T11:52:55.804 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:55.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.802+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45e80682f0 msgr2=0x7f45e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.802+0000 7f460abeb700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45e80682f0 0x7f45e806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f45f00048d0 tx=0x7f45f0009040 comp rx=0 tx=0).stop 2023-11-28T11:52:55.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.802+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 msgr2=0x7f46041f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:55.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.802+0000 7f460abeb700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 0x7f46041f5ee0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f45f8011aa0 tx=0x7f45f8011e60 comp rx=0 tx=0).stop 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.802+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 shutdown_connections 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.802+0000 7f460abeb700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46040c6420 0x7f46041f5ee0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.803+0000 7f460abeb700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45e80682f0 0x7f45e806a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.803+0000 7f460abeb700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46040c5830 0x7f46040ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.803+0000 7f460abeb700 1 --2- 172.21.15.144:0/576050743 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4604157d20 0x7f46040c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.803+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 >> 172.21.15.144:0/576050743 conn(0x7f46040d0000 msgr2=0x7f4604071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.803+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 shutdown_connections 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:55.803+0000 7f460abeb700 1 -- 172.21.15.144:0/576050743 wait complete. 2023-11-28T11:52:55.808 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:55.816 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-28T11:52:54.503286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71988,"kb_used_data":5044,"kb_used_omap":0,"kb_used_meta":66880,"kb_avail":754902732,"statfs":{"total":773094113280,"available":773020397568,"internally_reserved":0,"allocated":5165056,"data_stored":2835378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68485120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.695937"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.823962+0000","last_change":"2023-11-28T11:52:49.823962+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.823962+0000","last_undegraded":"2023-11-28T11:52:49.823962+0000","last_fullsized":"2023-11-28T11:52:49.823962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:11.475783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.824712+0000","last_change":"2023-11-28T11:52:49.824712+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.824712+0000","last_undegraded":"2023-11-28T11:52:49.824712+0000","last_fullsized":"2023-11-28T11:52:49.824712+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:37.366923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822634+0000","last_change":"2023-11-28T11:52:49.822634+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822634+0000","last_undegraded":"2023-11-28T11:52:49.822634+0000","last_fullsized":"2023-11-28T11:52:49.822634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:28.225138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822447+0000","last_change":"2023-11-28T11:52:49.822447+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822447+0000","last_undegraded":"2023-11-28T11:52:49.822447+0000","last_fullsized":"2023-11-28T11:52:49.822447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.465684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.822496+0000","last_change":"2023-11-28T11:52:49.822496+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.822496+0000","last_undegraded":"2023-11-28T11:52:49.822496+0000","last_fullsized":"2023-11-28T11:52:49.822496+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:54:06.429788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817298+0000","last_change":"2023-11-28T11:52:49.817298+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817298+0000","last_undegraded":"2023-11-28T11:52:49.817298+0000","last_fullsized":"2023-11-28T11:52:49.817298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:53.755885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.817657+0000","last_change":"2023-11-28T11:52:49.817657+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.817657+0000","last_undegraded":"2023-11-28T11:52:49.817657+0000","last_fullsized":"2023-11-28T11:52:49.817657+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:52.523879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815808+0000","last_change":"2023-11-28T11:52:49.815808+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815808+0000","last_undegraded":"2023-11-28T11:52:49.815808+0000","last_fullsized":"2023-11-28T11:52:49.815808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:00.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815142+0000","last_change":"2023-11-28T11:52:49.815142+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815142+0000","last_undegraded":"2023-11-28T11:52:49.815142+0000","last_fullsized":"2023-11-28T11:52:49.815142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:42.026408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.809482+0000","last_change":"2023-11-28T11:52:48.809482+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.809482+0000","last_undegraded":"2023-11-28T11:52:48.809482+0000","last_fullsized":"2023-11-28T11:52:48.809482+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:54.890718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813849+0000","last_change":"2023-11-28T11:52:48.813849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813849+0000","last_undegraded":"2023-11-28T11:52:48.813849+0000","last_fullsized":"2023-11-28T11:52:48.813849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813794+0000","last_change":"2023-11-28T11:52:48.813794+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813794+0000","last_undegraded":"2023-11-28T11:52:48.813794+0000","last_fullsized":"2023-11-28T11:52:48.813794+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815867+0000","last_change":"2023-11-28T11:52:48.815867+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815867+0000","last_undegraded":"2023-11-28T11:52:48.815867+0000","last_fullsized":"2023-11-28T11:52:48.815867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:02.973674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829812+0000","last_change":"2023-11-28T11:52:49.827097+0000","last_active":"2023-11-28T11:52:49.829812+0000","last_peered":"2023-11-28T11:52:49.829812+0000","last_clean":"2023-11-28T11:52:49.829812+0000","last_became_active":"2023-11-28T11:52:49.826826+0000","last_became_peered":"2023-11-28T11:52:49.826826+0000","last_unstale":"2023-11-28T11:52:49.829812+0000","last_undegraded":"2023-11-28T11:52:49.829812+0000","last_fullsized":"2023-11-28T11:52:49.829812+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:23.759602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811534+0000","last_change":"2023-11-28T11:52:48.811534+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811534+0000","last_undegraded":"2023-11-28T11:52:48.811534+0000","last_fullsized":"2023-11-28T11:52:48.811534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808962+0000","last_change":"2023-11-28T11:52:48.808962+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808962+0000","last_undegraded":"2023-11-28T11:52:48.808962+0000","last_fullsized":"2023-11-28T11:52:48.808962+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:22.861924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829014+0000","last_change":"2023-11-28T11:52:49.822322+0000","last_active":"2023-11-28T11:52:49.829014+0000","last_peered":"2023-11-28T11:52:49.829014+0000","last_clean":"2023-11-28T11:52:49.829014+0000","last_became_active":"2023-11-28T11:52:49.822077+0000","last_became_peered":"2023-11-28T11:52:49.822077+0000","last_unstale":"2023-11-28T11:52:49.829014+0000","last_undegraded":"2023-11-28T11:52:49.829014+0000","last_fullsized":"2023-11-28T11:52:49.829014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:24.019745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814558+0000","last_change":"2023-11-28T11:52:48.814558+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814558+0000","last_undegraded":"2023-11-28T11:52:48.814558+0000","last_fullsized":"2023-11-28T11:52:48.814558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:50.368503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.812849+0000","last_change":"2023-11-28T11:52:48.812849+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.812849+0000","last_undegraded":"2023-11-28T11:52:48.812849+0000","last_fullsized":"2023-11-28T11:52:48.812849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:22.526251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814234+0000","last_change":"2023-11-28T11:52:49.814234+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814234+0000","last_undegraded":"2023-11-28T11:52:49.814234+0000","last_fullsized":"2023-11-28T11:52:49.814234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:30.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.814475+0000","last_change":"2023-11-28T11:52:48.814475+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.814475+0000","last_undegraded":"2023-11-28T11:52:48.814475+0000","last_fullsized":"2023-11-28T11:52:48.814475+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:46.875258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810166+0000","last_change":"2023-11-28T11:52:48.810166+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810166+0000","last_undegraded":"2023-11-28T11:52:48.810166+0000","last_fullsized":"2023-11-28T11:52:48.810166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:16.135396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834400+0000","last_change":"2023-11-28T11:52:49.830260+0000","last_active":"2023-11-28T11:52:49.834400+0000","last_peered":"2023-11-28T11:52:49.834400+0000","last_clean":"2023-11-28T11:52:49.834400+0000","last_became_active":"2023-11-28T11:52:49.829977+0000","last_became_peered":"2023-11-28T11:52:49.829977+0000","last_unstale":"2023-11-28T11:52:49.834400+0000","last_undegraded":"2023-11-28T11:52:49.834400+0000","last_fullsized":"2023-11-28T11:52:49.834400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:19.733169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814190+0000","last_change":"2023-11-28T11:52:49.814190+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814190+0000","last_undegraded":"2023-11-28T11:52:49.814190+0000","last_fullsized":"2023-11-28T11:52:49.814190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:03.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813836+0000","last_change":"2023-11-28T11:52:48.813836+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813836+0000","last_undegraded":"2023-11-28T11:52:48.813836+0000","last_fullsized":"2023-11-28T11:52:48.813836+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.978385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.816827+0000","last_change":"2023-11-28T11:52:49.816827+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.816827+0000","last_undegraded":"2023-11-28T11:52:49.816827+0000","last_fullsized":"2023-11-28T11:52:49.816827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:30.533622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:44.118206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829400+0000","last_change":"2023-11-28T11:52:49.826554+0000","last_active":"2023-11-28T11:52:49.829400+0000","last_peered":"2023-11-28T11:52:49.829400+0000","last_clean":"2023-11-28T11:52:49.829400+0000","last_became_active":"2023-11-28T11:52:49.826299+0000","last_became_peered":"2023-11-28T11:52:49.826299+0000","last_unstale":"2023-11-28T11:52:49.829400+0000","last_undegraded":"2023-11-28T11:52:49.829400+0000","last_fullsized":"2023-11-28T11:52:49.829400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:33.111523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813679+0000","last_change":"2023-11-28T11:52:48.813679+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813679+0000","last_undegraded":"2023-11-28T11:52:48.813679+0000","last_fullsized":"2023-11-28T11:52:48.813679+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:51.984789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808979+0000","last_change":"2023-11-28T11:52:48.808979+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808979+0000","last_undegraded":"2023-11-28T11:52:48.808979+0000","last_fullsized":"2023-11-28T11:52:48.808979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808845+0000","last_change":"2023-11-28T11:52:48.808845+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808845+0000","last_undegraded":"2023-11-28T11:52:48.808845+0000","last_fullsized":"2023-11-28T11:52:48.808845+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:23.803406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813777+0000","last_change":"2023-11-28T11:52:48.813777+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813777+0000","last_undegraded":"2023-11-28T11:52:48.813777+0000","last_fullsized":"2023-11-28T11:52:48.813777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810666+0000","last_change":"2023-11-28T11:52:48.810666+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810666+0000","last_undegraded":"2023-11-28T11:52:48.810666+0000","last_fullsized":"2023-11-28T11:52:48.810666+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:54.122734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814746+0000","last_change":"2023-11-28T11:52:49.814746+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814746+0000","last_undegraded":"2023-11-28T11:52:49.814746+0000","last_fullsized":"2023-11-28T11:52:49.814746+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:26.868306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834375+0000","last_change":"2023-11-28T11:52:49.829671+0000","last_active":"2023-11-28T11:52:49.834375+0000","last_peered":"2023-11-28T11:52:49.834375+0000","last_clean":"2023-11-28T11:52:49.834375+0000","last_became_active":"2023-11-28T11:52:49.829404+0000","last_became_peered":"2023-11-28T11:52:49.829404+0000","last_unstale":"2023-11-28T11:52:49.834375+0000","last_undegraded":"2023-11-28T11:52:49.834375+0000","last_fullsized":"2023-11-28T11:52:49.834375+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:18.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813638+0000","last_change":"2023-11-28T11:52:48.813638+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813638+0000","last_undegraded":"2023-11-28T11:52:48.813638+0000","last_fullsized":"2023-11-28T11:52:48.813638+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:57.828731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.814357+0000","last_change":"2023-11-28T11:52:49.814357+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.814357+0000","last_undegraded":"2023-11-28T11:52:49.814357+0000","last_fullsized":"2023-11-28T11:52:49.814357+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:04.510511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810139+0000","last_change":"2023-11-28T11:52:48.810139+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810139+0000","last_undegraded":"2023-11-28T11:52:48.810139+0000","last_fullsized":"2023-11-28T11:52:48.810139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:39.463589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286150+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:46.286150+0000","last_peered":"2023-11-28T11:52:46.286150+0000","last_clean":"2023-11-28T11:52:46.286150+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:46.286150+0000","last_undegraded":"2023-11-28T11:52:46.286150+0000","last_fullsized":"2023-11-28T11:52:46.286150+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.810136+0000","last_change":"2023-11-28T11:52:48.810136+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.810136+0000","last_undegraded":"2023-11-28T11:52:48.810136+0000","last_fullsized":"2023-11-28T11:52:48.810136+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:33.980547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132093+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:46.132093+0000","last_peered":"2023-11-28T11:52:46.132093+0000","last_clean":"2023-11-28T11:52:46.132093+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:46.132093+0000","last_undegraded":"2023-11-28T11:52:46.132093+0000","last_fullsized":"2023-11-28T11:52:46.132093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.813768+0000","last_change":"2023-11-28T11:52:48.813768+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.813768+0000","last_undegraded":"2023-11-28T11:52:48.813768+0000","last_fullsized":"2023-11-28T11:52:48.813768+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:34.991301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.285986+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:46.285986+0000","last_peered":"2023-11-28T11:52:46.285986+0000","last_clean":"2023-11-28T11:52:46.285986+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:46.285986+0000","last_undegraded":"2023-11-28T11:52:46.285986+0000","last_fullsized":"2023-11-28T11:52:46.285986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":181,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132872+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:46.132872+0000","last_peered":"2023-11-28T11:52:46.132872+0000","last_clean":"2023-11-28T11:52:46.132872+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:46.132872+0000","last_undegraded":"2023-11-28T11:52:46.132872+0000","last_fullsized":"2023-11-28T11:52:46.132872+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.829752+0000","last_change":"2023-11-28T11:52:49.824383+0000","last_active":"2023-11-28T11:52:49.829752+0000","last_peered":"2023-11-28T11:52:49.829752+0000","last_clean":"2023-11-28T11:52:49.829752+0000","last_became_active":"2023-11-28T11:52:49.824081+0000","last_became_peered":"2023-11-28T11:52:49.824081+0000","last_unstale":"2023-11-28T11:52:49.829752+0000","last_undegraded":"2023-11-28T11:52:49.829752+0000","last_fullsized":"2023-11-28T11:52:49.829752+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:17.464586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132870+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:46.132870+0000","last_peered":"2023-11-28T11:52:46.132870+0000","last_clean":"2023-11-28T11:52:46.132870+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:46.132870+0000","last_undegraded":"2023-11-28T11:52:46.132870+0000","last_fullsized":"2023-11-28T11:52:46.132870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-28T11:52:49.834419+0000","last_change":"2023-11-28T11:52:49.830269+0000","last_active":"2023-11-28T11:52:49.834419+0000","last_peered":"2023-11-28T11:52:49.834419+0000","last_clean":"2023-11-28T11:52:49.834419+0000","last_became_active":"2023-11-28T11:52:49.830011+0000","last_became_peered":"2023-11-28T11:52:49.830011+0000","last_unstale":"2023-11-28T11:52:49.834419+0000","last_undegraded":"2023-11-28T11:52:49.834419+0000","last_fullsized":"2023-11-28T11:52:49.834419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:05.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.132859+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:46.132859+0000","last_peered":"2023-11-28T11:52:46.132859+0000","last_clean":"2023-11-28T11:52:46.132859+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:46.132859+0000","last_undegraded":"2023-11-28T11:52:46.132859+0000","last_fullsized":"2023-11-28T11:52:46.132859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.808847+0000","last_change":"2023-11-28T11:52:48.808847+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.808847+0000","last_undegraded":"2023-11-28T11:52:48.808847+0000","last_fullsized":"2023-11-28T11:52:48.808847+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.286133+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:46.286133+0000","last_peered":"2023-11-28T11:52:46.286133+0000","last_clean":"2023-11-28T11:52:46.286133+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:46.286133+0000","last_undegraded":"2023-11-28T11:52:46.286133+0000","last_fullsized":"2023-11-28T11:52:46.286133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:52:49.804716+0000","last_change":"2023-11-28T11:52:49.804716+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.804716+0000","last_undegraded":"2023-11-28T11:52:49.804716+0000","last_fullsized":"2023-11-28T11:52:49.804716+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.815863+0000","last_change":"2023-11-28T11:52:48.815863+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.815863+0000","last_undegraded":"2023-11-28T11:52:48.815863+0000","last_fullsized":"2023-11-28T11:52:48.815863+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131458+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:46.131458+0000","last_peered":"2023-11-28T11:52:46.131458+0000","last_clean":"2023-11-28T11:52:46.131458+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:46.131458+0000","last_undegraded":"2023-11-28T11:52:46.131458+0000","last_fullsized":"2023-11-28T11:52:46.131458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-11-28T11:52:49.815176+0000","last_change":"2023-11-28T11:52:49.815176+0000","last_active":"2023-11-28T11:52:49.804716+0000","last_peered":"2023-11-28T11:52:49.804716+0000","last_clean":"2023-11-28T11:52:49.804716+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:49.815176+0000","last_undegraded":"2023-11-28T11:52:49.815176+0000","last_fullsized":"2023-11-28T11:52:49.815176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:55.616287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-28T11:52:48.811601+0000","last_change":"2023-11-28T11:52:48.811601+0000","last_active":"2023-11-28T11:52:48.796034+0000","last_peered":"2023-11-28T11:52:48.796034+0000","last_clean":"2023-11-28T11:52:48.796034+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:52:48.811601+0000","last_undegraded":"2023-11-28T11:52:48.811601+0000","last_fullsized":"2023-11-28T11:52:48.811601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:48:34.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:52:46.131493+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:46.131493+0000","last_peered":"2023-11-28T11:52:46.131493+0000","last_clean":"2023-11-28T11:52:46.131493+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:46.131493+0000","last_undegraded":"2023-11-28T11:52:46.131493+0000","last_fullsized":"2023-11-28T11:52:46.131493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9896,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361944,"statfs":{"total":96636764160,"available":96626630656,"internally_reserved":0,"allocated":1081344,"data_stored":792046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59899999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":3.7810000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.753},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64400000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59699999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9716,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362124,"statfs":{"total":96636764160,"available":96626814976,"internally_reserved":0,"allocated":1093632,"data_stored":798216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.27400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.30199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.63}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75700000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8684,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363156,"statfs":{"total":96636764160,"available":96627871744,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.317},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.32900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.498}]},{"osd":5,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8624,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363216,"statfs":{"total":96636764160,"available":96627933184,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.317}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.501}]}]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8880,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362960,"statfs":{"total":96636764160,"available":96627671040,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.254},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.30199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":16.231000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.755},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60199999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":207848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.47599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.441},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8748,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363092,"statfs":{"total":96636764160,"available":96627806208,"internally_reserved":0,"allocated":495616,"data_stored":206862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:58.822 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:52:58.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.953+0000 7f72dd354700 1 Processor -- start 2023-11-28T11:52:58.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.953+0000 7f72dd354700 1 -- start start 2023-11-28T11:52:58.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72dd354700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f72d80cd8c0 0x7f72d8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:58.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72dd354700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cdc90 0x7f72d80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:58.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72dd354700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d813ff00 con 0x7f72d80b2e70 2023-11-28T11:52:58.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72dd354700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d81404c0 con 0x7f72d80cd8c0 2023-11-28T11:52:58.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72dd354700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d815aa80 con 0x7f72d80cdc90 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72d6ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f72d80cd8c0 0x7f72d8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72d77fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f72d80b2e70 0x7f72d806f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54186/0 (socket says 172.21.15.144:54186) 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.954+0000 7f72d77fe700 1 -- 172.21.15.144:0/310432481 learned_addr learned my addr 172.21.15.144:0/310432481 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d67fc700 1 --2- 172.21.15.144:0/310432481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cdc90 0x7f72d80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d6ffd700 1 -- 172.21.15.144:0/310432481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cdc90 msgr2=0x7f72d80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d6ffd700 1 --2- 172.21.15.144:0/310432481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cdc90 0x7f72d80b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d6ffd700 1 -- 172.21.15.144:0/310432481 >> v1:172.21.15.144:6789/0 conn(0x7f72d80b2e70 legacy=0x7f72d806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d6ffd700 1 -- 172.21.15.144:0/310432481 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80b3310 con 0x7f72d80cd8c0 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d5ffb700 1 -- 172.21.15.144:0/310432481 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307698733 0 0) 0x7f72d813ff00 con 0x7f72d80b2e70 2023-11-28T11:52:58.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d6ffd700 1 --2- 172.21.15.144:0/310432481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cd8c0 0x7f72d8140950 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009820 tx=0x7f72c8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ffda0fc68eb81d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d5ffb700 1 -- 172.21.15.144:0/310432481 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8005e30 con 0x7f72d80cd8c0 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d5ffb700 1 -- 172.21.15.144:0/310432481 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8006030 con 0x7f72d80cd8c0 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.955+0000 7f72d5ffb700 1 -- 172.21.15.144:0/310432481 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c80063e0 con 0x7f72d80cd8c0 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 -- 172.21.15.144:0/310432481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cd8c0 msgr2=0x7f72d8140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 --2- 172.21.15.144:0/310432481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cd8c0 0x7f72d8140950 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009820 tx=0x7f72c8009b30 comp rx=0 tx=0).stop 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 -- 172.21.15.144:0/310432481 shutdown_connections 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 --2- 172.21.15.144:0/310432481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cd8c0 0x7f72d8140950 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 --2- 172.21.15.144:0/310432481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cdc90 0x7f72d80b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:58.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 -- 172.21.15.144:0/310432481 >> 172.21.15.144:0/310432481 conn(0x7f72d806aed0 msgr2=0x7f72d806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:58.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 -- 172.21.15.144:0/310432481 shutdown_connections 2023-11-28T11:52:58.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.956+0000 7f72dd354700 1 -- 172.21.15.144:0/310432481 wait complete. 2023-11-28T11:52:58.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.957+0000 7f72dd354700 1 Processor -- start 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.957+0000 7f72dd354700 1 -- start start 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72dd354700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72d80b2e70 0x7f72d80c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72dd354700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cd8c0 0x7f72d80b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72dd354700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 0x7f72d80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72dd354700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80c8db0 con 0x7f72d80cdc90 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72dd354700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80c5490 con 0x7f72d80b2e70 2023-11-28T11:52:58.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72dd354700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80c4b60 con 0x7f72d80cd8c0 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 0x7f72d80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 0x7f72d80c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37480/0 (socket says 172.21.15.144:37480) 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 -- 172.21.15.144:0/233586716 learned_addr learned my addr 172.21.15.144:0/233586716 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d67fc700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cd8c0 0x7f72d80b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d6ffd700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72d80b2e70 0x7f72d80c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 -- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cd8c0 msgr2=0x7f72d80b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:58.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cd8c0 0x7f72d80b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 -- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72d80b2e70 msgr2=0x7f72d80c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72d80b2e70 0x7f72d80c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 -- 172.21.15.144:0/233586716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c8009430 con 0x7f72d80cdc90 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.958+0000 7f72d77fe700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 0x7f72d80c8870 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72d000cb40 tx=0x7f72d000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.959+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000ab50 con 0x7f72d80cdc90 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.959+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d000acd0 con 0x7f72d80cdc90 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.959+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d8140150 con 0x7f72d80cdc90 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.959+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0007310 con 0x7f72d80cdc90 2023-11-28T11:52:58.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.959+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d80b35f0 con 0x7f72d80cdc90 2023-11-28T11:52:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.960+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d80ce100 con 0x7f72d80cdc90 2023-11-28T11:52:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.960+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f72d001a030 con 0x7f72d80cdc90 2023-11-28T11:52:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.960+0000 7f72bf7fe700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f72c40682f0 0x7f72c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.961+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72d0085980 con 0x7f72d80cdc90 2023-11-28T11:52:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.961+0000 7f72d6ffd700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f72c40682f0 0x7f72c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:58.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.961+0000 7f72d6ffd700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f72c40682f0 0x7f72c406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f72c8013040 tx=0x7f72c8008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:58.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:58.965+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f72d0052180 con 0x7f72d80cdc90 2023-11-28T11:52:59.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.142+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72d8068690 con 0x7f72c40682f0 2023-11-28T11:52:59.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.150+0000 7f72bf7fe700 1 -- 172.21.15.144:0/233586716 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311873 (secure 0 0 0) 0x7f72d8068690 con 0x7f72c40682f0 2023-11-28T11:52:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f72c40682f0 msgr2=0x7f72c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f72c40682f0 0x7f72c406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f72c8013040 tx=0x7f72c8008070 comp rx=0 tx=0).stop 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 msgr2=0x7f72d80c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 0x7f72d80c8870 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72d000cb40 tx=0x7f72d000cf00 comp rx=0 tx=0).stop 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 shutdown_connections 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72d80cdc90 0x7f72d80c8870 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f72c40682f0 0x7f72c406a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72d80b2e70 0x7f72d80c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 --2- 172.21.15.144:0/233586716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72d80cd8c0 0x7f72d80b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 >> 172.21.15.144:0/233586716 conn(0x7f72d806aed0 msgr2=0x7f72d813eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 shutdown_connections 2023-11-28T11:52:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.154+0000 7f72dd354700 1 -- 172.21.15.144:0/233586716 wait complete. 2023-11-28T11:52:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:52:59.168 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-11-28T11:52:58.504466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":73632,"kb_used_data":5408,"kb_used_omap":0,"kb_used_meta":68160,"kb_avail":754901088,"statfs":{"total":773094113280,"available":773018714112,"internally_reserved":0,"allocated":5537792,"data_stored":3009480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":69795840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.294938"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505125+0000","last_change":"2023-11-28T11:52:50.223862+0000","last_active":"2023-11-28T11:52:51.505125+0000","last_peered":"2023-11-28T11:52:51.505125+0000","last_clean":"2023-11-28T11:52:51.505125+0000","last_became_active":"2023-11-28T11:52:50.223614+0000","last_became_peered":"2023-11-28T11:52:50.223614+0000","last_unstale":"2023-11-28T11:52:51.505125+0000","last_undegraded":"2023-11-28T11:52:51.505125+0000","last_fullsized":"2023-11-28T11:52:51.505125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:35.999983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505922+0000","last_change":"2023-11-28T11:52:50.225799+0000","last_active":"2023-11-28T11:52:51.505922+0000","last_peered":"2023-11-28T11:52:51.505922+0000","last_clean":"2023-11-28T11:52:51.505922+0000","last_became_active":"2023-11-28T11:52:50.225126+0000","last_became_peered":"2023-11-28T11:52:50.225126+0000","last_unstale":"2023-11-28T11:52:51.505922+0000","last_undegraded":"2023-11-28T11:52:51.505922+0000","last_fullsized":"2023-11-28T11:52:51.505922+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:17.791185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.240099+0000","last_change":"2023-11-28T11:52:50.236411+0000","last_active":"2023-11-28T11:52:50.240099+0000","last_peered":"2023-11-28T11:52:50.240099+0000","last_clean":"2023-11-28T11:52:50.240099+0000","last_became_active":"2023-11-28T11:52:50.236182+0000","last_became_peered":"2023-11-28T11:52:50.236182+0000","last_unstale":"2023-11-28T11:52:50.240099+0000","last_undegraded":"2023-11-28T11:52:50.240099+0000","last_fullsized":"2023-11-28T11:52:50.240099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:11.475783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505792+0000","last_change":"2023-11-28T11:52:50.223370+0000","last_active":"2023-11-28T11:52:51.505792+0000","last_peered":"2023-11-28T11:52:51.505792+0000","last_clean":"2023-11-28T11:52:51.505792+0000","last_became_active":"2023-11-28T11:52:50.223091+0000","last_became_peered":"2023-11-28T11:52:50.223091+0000","last_unstale":"2023-11-28T11:52:51.505792+0000","last_undegraded":"2023-11-28T11:52:51.505792+0000","last_fullsized":"2023-11-28T11:52:51.505792+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:58.036929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505206+0000","last_change":"2023-11-28T11:52:50.230363+0000","last_active":"2023-11-28T11:52:51.505206+0000","last_peered":"2023-11-28T11:52:51.505206+0000","last_clean":"2023-11-28T11:52:51.505206+0000","last_became_active":"2023-11-28T11:52:50.230094+0000","last_became_peered":"2023-11-28T11:52:50.230094+0000","last_unstale":"2023-11-28T11:52:51.505206+0000","last_undegraded":"2023-11-28T11:52:51.505206+0000","last_fullsized":"2023-11-28T11:52:51.505206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:11:57.360530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504234+0000","last_change":"2023-11-28T11:52:50.217969+0000","last_active":"2023-11-28T11:52:51.504234+0000","last_peered":"2023-11-28T11:52:51.504234+0000","last_clean":"2023-11-28T11:52:51.504234+0000","last_became_active":"2023-11-28T11:52:50.217735+0000","last_became_peered":"2023-11-28T11:52:50.217735+0000","last_unstale":"2023-11-28T11:52:51.504234+0000","last_undegraded":"2023-11-28T11:52:51.504234+0000","last_fullsized":"2023-11-28T11:52:51.504234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:48:35.069385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.242427+0000","last_change":"2023-11-28T11:52:50.237081+0000","last_active":"2023-11-28T11:52:50.242427+0000","last_peered":"2023-11-28T11:52:50.242427+0000","last_clean":"2023-11-28T11:52:50.242427+0000","last_became_active":"2023-11-28T11:52:50.236525+0000","last_became_peered":"2023-11-28T11:52:50.236525+0000","last_unstale":"2023-11-28T11:52:50.242427+0000","last_undegraded":"2023-11-28T11:52:50.242427+0000","last_fullsized":"2023-11-28T11:52:50.242427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:37.366923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221068+0000","last_change":"2023-11-28T11:52:50.222498+0000","last_active":"2023-11-28T11:52:51.221068+0000","last_peered":"2023-11-28T11:52:51.221068+0000","last_clean":"2023-11-28T11:52:51.221068+0000","last_became_active":"2023-11-28T11:52:50.222259+0000","last_became_peered":"2023-11-28T11:52:50.222259+0000","last_unstale":"2023-11-28T11:52:51.221068+0000","last_undegraded":"2023-11-28T11:52:51.221068+0000","last_fullsized":"2023-11-28T11:52:51.221068+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:14.198398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221339+0000","last_change":"2023-11-28T11:52:50.222472+0000","last_active":"2023-11-28T11:52:51.221339+0000","last_peered":"2023-11-28T11:52:51.221339+0000","last_clean":"2023-11-28T11:52:51.221339+0000","last_became_active":"2023-11-28T11:52:50.222215+0000","last_became_peered":"2023-11-28T11:52:50.222215+0000","last_unstale":"2023-11-28T11:52:51.221339+0000","last_undegraded":"2023-11-28T11:52:51.221339+0000","last_fullsized":"2023-11-28T11:52:51.221339+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:14.957517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505022+0000","last_change":"2023-11-28T11:52:50.224922+0000","last_active":"2023-11-28T11:52:51.505022+0000","last_peered":"2023-11-28T11:52:51.505022+0000","last_clean":"2023-11-28T11:52:51.505022+0000","last_became_active":"2023-11-28T11:52:50.224345+0000","last_became_peered":"2023-11-28T11:52:50.224345+0000","last_unstale":"2023-11-28T11:52:51.505022+0000","last_undegraded":"2023-11-28T11:52:51.505022+0000","last_fullsized":"2023-11-28T11:52:51.505022+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:31.880630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.218580+0000","last_change":"2023-11-28T11:52:50.218643+0000","last_active":"2023-11-28T11:52:51.218580+0000","last_peered":"2023-11-28T11:52:51.218580+0000","last_clean":"2023-11-28T11:52:51.218580+0000","last_became_active":"2023-11-28T11:52:50.218401+0000","last_became_peered":"2023-11-28T11:52:50.218401+0000","last_unstale":"2023-11-28T11:52:51.218580+0000","last_undegraded":"2023-11-28T11:52:51.218580+0000","last_fullsized":"2023-11-28T11:52:51.218580+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:59.604990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221394+0000","last_change":"2023-11-28T11:52:50.230783+0000","last_active":"2023-11-28T11:52:51.221394+0000","last_peered":"2023-11-28T11:52:51.221394+0000","last_clean":"2023-11-28T11:52:51.221394+0000","last_became_active":"2023-11-28T11:52:50.229631+0000","last_became_peered":"2023-11-28T11:52:50.229631+0000","last_unstale":"2023-11-28T11:52:51.221394+0000","last_undegraded":"2023-11-28T11:52:51.221394+0000","last_fullsized":"2023-11-28T11:52:51.221394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:16.119194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504185+0000","last_change":"2023-11-28T11:52:50.225792+0000","last_active":"2023-11-28T11:52:51.504185+0000","last_peered":"2023-11-28T11:52:51.504185+0000","last_clean":"2023-11-28T11:52:51.504185+0000","last_became_active":"2023-11-28T11:52:50.225575+0000","last_became_peered":"2023-11-28T11:52:50.225575+0000","last_unstale":"2023-11-28T11:52:51.504185+0000","last_undegraded":"2023-11-28T11:52:51.504185+0000","last_fullsized":"2023-11-28T11:52:51.504185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:23.534933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.240142+0000","last_change":"2023-11-28T11:52:50.236399+0000","last_active":"2023-11-28T11:52:50.240142+0000","last_peered":"2023-11-28T11:52:50.240142+0000","last_clean":"2023-11-28T11:52:50.240142+0000","last_became_active":"2023-11-28T11:52:50.236128+0000","last_became_peered":"2023-11-28T11:52:50.236128+0000","last_unstale":"2023-11-28T11:52:50.240142+0000","last_undegraded":"2023-11-28T11:52:50.240142+0000","last_fullsized":"2023-11-28T11:52:50.240142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:28.225138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505549+0000","last_change":"2023-11-28T11:52:50.227035+0000","last_active":"2023-11-28T11:52:51.505549+0000","last_peered":"2023-11-28T11:52:51.505549+0000","last_clean":"2023-11-28T11:52:51.505549+0000","last_became_active":"2023-11-28T11:52:50.226771+0000","last_became_peered":"2023-11-28T11:52:50.226771+0000","last_unstale":"2023-11-28T11:52:51.505549+0000","last_undegraded":"2023-11-28T11:52:51.505549+0000","last_fullsized":"2023-11-28T11:52:51.505549+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:35.432520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504106+0000","last_change":"2023-11-28T11:52:50.225142+0000","last_active":"2023-11-28T11:52:51.504106+0000","last_peered":"2023-11-28T11:52:51.504106+0000","last_clean":"2023-11-28T11:52:51.504106+0000","last_became_active":"2023-11-28T11:52:50.224699+0000","last_became_peered":"2023-11-28T11:52:50.224699+0000","last_unstale":"2023-11-28T11:52:51.504106+0000","last_undegraded":"2023-11-28T11:52:51.504106+0000","last_fullsized":"2023-11-28T11:52:51.504106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:51.377185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.234592+0000","last_change":"2023-11-28T11:52:50.232504+0000","last_active":"2023-11-28T11:52:50.234592+0000","last_peered":"2023-11-28T11:52:50.234592+0000","last_clean":"2023-11-28T11:52:50.234592+0000","last_became_active":"2023-11-28T11:52:50.230170+0000","last_became_peered":"2023-11-28T11:52:50.230170+0000","last_unstale":"2023-11-28T11:52:50.234592+0000","last_undegraded":"2023-11-28T11:52:50.234592+0000","last_fullsized":"2023-11-28T11:52:50.234592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.465684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.506066+0000","last_change":"2023-11-28T11:52:50.225657+0000","last_active":"2023-11-28T11:52:51.506066+0000","last_peered":"2023-11-28T11:52:51.506066+0000","last_clean":"2023-11-28T11:52:51.506066+0000","last_became_active":"2023-11-28T11:52:50.224773+0000","last_became_peered":"2023-11-28T11:52:50.224773+0000","last_unstale":"2023-11-28T11:52:51.506066+0000","last_undegraded":"2023-11-28T11:52:51.506066+0000","last_fullsized":"2023-11-28T11:52:51.506066+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:17.144078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.231777+0000","last_change":"2023-11-28T11:52:50.227206+0000","last_active":"2023-11-28T11:52:50.231777+0000","last_peered":"2023-11-28T11:52:50.231777+0000","last_clean":"2023-11-28T11:52:50.231777+0000","last_became_active":"2023-11-28T11:52:50.226954+0000","last_became_peered":"2023-11-28T11:52:50.226954+0000","last_unstale":"2023-11-28T11:52:50.231777+0000","last_undegraded":"2023-11-28T11:52:50.231777+0000","last_fullsized":"2023-11-28T11:52:50.231777+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:54:06.429788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.220191+0000","last_change":"2023-11-28T11:52:50.237099+0000","last_active":"2023-11-28T11:52:51.220191+0000","last_peered":"2023-11-28T11:52:51.220191+0000","last_clean":"2023-11-28T11:52:51.220191+0000","last_became_active":"2023-11-28T11:52:50.236854+0000","last_became_peered":"2023-11-28T11:52:50.236854+0000","last_unstale":"2023-11-28T11:52:51.220191+0000","last_undegraded":"2023-11-28T11:52:51.220191+0000","last_fullsized":"2023-11-28T11:52:51.220191+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:03.982356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.220028+0000","last_change":"2023-11-28T11:52:50.232719+0000","last_active":"2023-11-28T11:52:51.220028+0000","last_peered":"2023-11-28T11:52:51.220028+0000","last_clean":"2023-11-28T11:52:51.220028+0000","last_became_active":"2023-11-28T11:52:50.232438+0000","last_became_peered":"2023-11-28T11:52:50.232438+0000","last_unstale":"2023-11-28T11:52:51.220028+0000","last_undegraded":"2023-11-28T11:52:51.220028+0000","last_fullsized":"2023-11-28T11:52:51.220028+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:51.377185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221092+0000","last_change":"2023-11-28T11:52:50.229546+0000","last_active":"2023-11-28T11:52:51.221092+0000","last_peered":"2023-11-28T11:52:51.221092+0000","last_clean":"2023-11-28T11:52:51.221092+0000","last_became_active":"2023-11-28T11:52:50.229183+0000","last_became_peered":"2023-11-28T11:52:50.229183+0000","last_unstale":"2023-11-28T11:52:51.221092+0000","last_undegraded":"2023-11-28T11:52:51.221092+0000","last_fullsized":"2023-11-28T11:52:51.221092+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:00.779877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.218368+0000","last_change":"2023-11-28T11:52:50.219085+0000","last_active":"2023-11-28T11:52:51.218368+0000","last_peered":"2023-11-28T11:52:51.218368+0000","last_clean":"2023-11-28T11:52:51.218368+0000","last_became_active":"2023-11-28T11:52:50.218837+0000","last_became_peered":"2023-11-28T11:52:50.218837+0000","last_unstale":"2023-11-28T11:52:51.218368+0000","last_undegraded":"2023-11-28T11:52:51.218368+0000","last_fullsized":"2023-11-28T11:52:51.218368+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:55.287759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.240034+0000","last_change":"2023-11-28T11:52:50.236274+0000","last_active":"2023-11-28T11:52:50.240034+0000","last_peered":"2023-11-28T11:52:50.240034+0000","last_clean":"2023-11-28T11:52:50.240034+0000","last_became_active":"2023-11-28T11:52:50.235988+0000","last_became_peered":"2023-11-28T11:52:50.235988+0000","last_unstale":"2023-11-28T11:52:50.240034+0000","last_undegraded":"2023-11-28T11:52:50.240034+0000","last_fullsized":"2023-11-28T11:52:50.240034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:53.755885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.232781+0000","last_change":"2023-11-28T11:52:50.228301+0000","last_active":"2023-11-28T11:52:50.232781+0000","last_peered":"2023-11-28T11:52:50.232781+0000","last_clean":"2023-11-28T11:52:50.232781+0000","last_became_active":"2023-11-28T11:52:50.228049+0000","last_became_peered":"2023-11-28T11:52:50.228049+0000","last_unstale":"2023-11-28T11:52:50.232781+0000","last_undegraded":"2023-11-28T11:52:50.232781+0000","last_fullsized":"2023-11-28T11:52:50.232781+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:52.523879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221473+0000","last_change":"2023-11-28T11:52:50.229352+0000","last_active":"2023-11-28T11:52:51.221473+0000","last_peered":"2023-11-28T11:52:51.221473+0000","last_clean":"2023-11-28T11:52:51.221473+0000","last_became_active":"2023-11-28T11:52:50.229059+0000","last_became_peered":"2023-11-28T11:52:50.229059+0000","last_unstale":"2023-11-28T11:52:51.221473+0000","last_undegraded":"2023-11-28T11:52:51.221473+0000","last_fullsized":"2023-11-28T11:52:51.221473+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:33:11.065389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505145+0000","last_change":"2023-11-28T11:52:50.228420+0000","last_active":"2023-11-28T11:52:51.505145+0000","last_peered":"2023-11-28T11:52:51.505145+0000","last_clean":"2023-11-28T11:52:51.505145+0000","last_became_active":"2023-11-28T11:52:50.228149+0000","last_became_peered":"2023-11-28T11:52:50.228149+0000","last_unstale":"2023-11-28T11:52:51.505145+0000","last_undegraded":"2023-11-28T11:52:51.505145+0000","last_fullsized":"2023-11-28T11:52:51.505145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:45.331162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.220249+0000","last_change":"2023-11-28T11:52:50.234318+0000","last_active":"2023-11-28T11:52:51.220249+0000","last_peered":"2023-11-28T11:52:51.220249+0000","last_clean":"2023-11-28T11:52:51.220249+0000","last_became_active":"2023-11-28T11:52:50.234032+0000","last_became_peered":"2023-11-28T11:52:50.234032+0000","last_unstale":"2023-11-28T11:52:51.220249+0000","last_undegraded":"2023-11-28T11:52:51.220249+0000","last_fullsized":"2023-11-28T11:52:51.220249+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:14.502256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.240029+0000","last_change":"2023-11-28T11:52:50.235204+0000","last_active":"2023-11-28T11:52:50.240029+0000","last_peered":"2023-11-28T11:52:50.240029+0000","last_clean":"2023-11-28T11:52:50.240029+0000","last_became_active":"2023-11-28T11:52:50.234965+0000","last_became_peered":"2023-11-28T11:52:50.234965+0000","last_unstale":"2023-11-28T11:52:50.240029+0000","last_undegraded":"2023-11-28T11:52:50.240029+0000","last_fullsized":"2023-11-28T11:52:50.240029+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:00.114504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505714+0000","last_change":"2023-11-28T11:52:50.222676+0000","last_active":"2023-11-28T11:52:51.505714+0000","last_peered":"2023-11-28T11:52:51.505714+0000","last_clean":"2023-11-28T11:52:51.505714+0000","last_became_active":"2023-11-28T11:52:50.221903+0000","last_became_peered":"2023-11-28T11:52:50.221903+0000","last_unstale":"2023-11-28T11:52:51.505714+0000","last_undegraded":"2023-11-28T11:52:51.505714+0000","last_fullsized":"2023-11-28T11:52:51.505714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:11:57.360530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.242446+0000","last_change":"2023-11-28T11:52:50.237240+0000","last_active":"2023-11-28T11:52:50.242446+0000","last_peered":"2023-11-28T11:52:50.242446+0000","last_clean":"2023-11-28T11:52:50.242446+0000","last_became_active":"2023-11-28T11:52:50.236831+0000","last_became_peered":"2023-11-28T11:52:50.236831+0000","last_unstale":"2023-11-28T11:52:50.242446+0000","last_undegraded":"2023-11-28T11:52:50.242446+0000","last_fullsized":"2023-11-28T11:52:50.242446+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:42.026408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504987+0000","last_change":"2023-11-28T11:52:50.229529+0000","last_active":"2023-11-28T11:52:51.504987+0000","last_peered":"2023-11-28T11:52:51.504987+0000","last_clean":"2023-11-28T11:52:51.504987+0000","last_became_active":"2023-11-28T11:52:50.229234+0000","last_became_peered":"2023-11-28T11:52:50.229234+0000","last_unstale":"2023-11-28T11:52:51.504987+0000","last_undegraded":"2023-11-28T11:52:51.504987+0000","last_fullsized":"2023-11-28T11:52:51.504987+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:23.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504782+0000","last_change":"2023-11-28T11:52:50.225074+0000","last_active":"2023-11-28T11:52:51.504782+0000","last_peered":"2023-11-28T11:52:51.504782+0000","last_clean":"2023-11-28T11:52:51.504782+0000","last_became_active":"2023-11-28T11:52:50.224798+0000","last_became_peered":"2023-11-28T11:52:50.224798+0000","last_unstale":"2023-11-28T11:52:51.504782+0000","last_undegraded":"2023-11-28T11:52:51.504782+0000","last_fullsized":"2023-11-28T11:52:51.504782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:40.472271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"45'4","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.237365+0000","last_change":"2023-11-28T11:52:49.822569+0000","last_active":"2023-11-28T11:52:50.237365+0000","last_peered":"2023-11-28T11:52:50.237365+0000","last_clean":"2023-11-28T11:52:50.237365+0000","last_became_active":"2023-11-28T11:52:49.821894+0000","last_became_peered":"2023-11-28T11:52:49.821894+0000","last_unstale":"2023-11-28T11:52:50.237365+0000","last_undegraded":"2023-11-28T11:52:50.237365+0000","last_fullsized":"2023-11-28T11:52:50.237365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:54.890718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505016+0000","last_change":"2023-11-28T11:52:50.230249+0000","last_active":"2023-11-28T11:52:51.505016+0000","last_peered":"2023-11-28T11:52:51.505016+0000","last_clean":"2023-11-28T11:52:51.505016+0000","last_became_active":"2023-11-28T11:52:50.229966+0000","last_became_peered":"2023-11-28T11:52:50.229966+0000","last_unstale":"2023-11-28T11:52:51.505016+0000","last_undegraded":"2023-11-28T11:52:51.505016+0000","last_fullsized":"2023-11-28T11:52:51.505016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:11:57.360530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.233128+0000","last_change":"2023-11-28T11:52:49.828472+0000","last_active":"2023-11-28T11:52:50.233128+0000","last_peered":"2023-11-28T11:52:50.233128+0000","last_clean":"2023-11-28T11:52:50.233128+0000","last_became_active":"2023-11-28T11:52:49.828237+0000","last_became_peered":"2023-11-28T11:52:49.828237+0000","last_unstale":"2023-11-28T11:52:50.233128+0000","last_undegraded":"2023-11-28T11:52:50.233128+0000","last_fullsized":"2023-11-28T11:52:50.233128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.219975+0000","last_change":"2023-11-28T11:52:50.233728+0000","last_active":"2023-11-28T11:52:51.219975+0000","last_peered":"2023-11-28T11:52:51.219975+0000","last_clean":"2023-11-28T11:52:51.219975+0000","last_became_active":"2023-11-28T11:52:50.233447+0000","last_became_peered":"2023-11-28T11:52:50.233447+0000","last_unstale":"2023-11-28T11:52:51.219975+0000","last_undegraded":"2023-11-28T11:52:51.219975+0000","last_fullsized":"2023-11-28T11:52:51.219975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:47.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.238090+0000","last_change":"2023-11-28T11:52:49.827440+0000","last_active":"2023-11-28T11:52:50.238090+0000","last_peered":"2023-11-28T11:52:50.238090+0000","last_clean":"2023-11-28T11:52:50.238090+0000","last_became_active":"2023-11-28T11:52:49.826040+0000","last_became_peered":"2023-11-28T11:52:49.826040+0000","last_unstale":"2023-11-28T11:52:50.238090+0000","last_undegraded":"2023-11-28T11:52:50.238090+0000","last_fullsized":"2023-11-28T11:52:50.238090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221326+0000","last_change":"2023-11-28T11:52:50.229772+0000","last_active":"2023-11-28T11:52:51.221326+0000","last_peered":"2023-11-28T11:52:51.221326+0000","last_clean":"2023-11-28T11:52:51.221326+0000","last_became_active":"2023-11-28T11:52:50.229083+0000","last_became_peered":"2023-11-28T11:52:50.229083+0000","last_unstale":"2023-11-28T11:52:51.221326+0000","last_undegraded":"2023-11-28T11:52:51.221326+0000","last_fullsized":"2023-11-28T11:52:51.221326+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:46.760771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.231259+0000","last_change":"2023-11-28T11:52:49.825507+0000","last_active":"2023-11-28T11:52:50.231259+0000","last_peered":"2023-11-28T11:52:50.231259+0000","last_clean":"2023-11-28T11:52:50.231259+0000","last_became_active":"2023-11-28T11:52:49.825322+0000","last_became_peered":"2023-11-28T11:52:49.825322+0000","last_unstale":"2023-11-28T11:52:50.231259+0000","last_undegraded":"2023-11-28T11:52:50.231259+0000","last_fullsized":"2023-11-28T11:52:50.231259+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:02.973674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505382+0000","last_change":"2023-11-28T11:52:50.225142+0000","last_active":"2023-11-28T11:52:51.505382+0000","last_peered":"2023-11-28T11:52:51.505382+0000","last_clean":"2023-11-28T11:52:51.505382+0000","last_became_active":"2023-11-28T11:52:50.224716+0000","last_became_peered":"2023-11-28T11:52:50.224716+0000","last_unstale":"2023-11-28T11:52:51.505382+0000","last_undegraded":"2023-11-28T11:52:51.505382+0000","last_fullsized":"2023-11-28T11:52:51.505382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:07.381326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.235900+0000","last_change":"2023-11-28T11:52:49.827097+0000","last_active":"2023-11-28T11:52:50.235900+0000","last_peered":"2023-11-28T11:52:50.235900+0000","last_clean":"2023-11-28T11:52:50.235900+0000","last_became_active":"2023-11-28T11:52:49.826826+0000","last_became_peered":"2023-11-28T11:52:49.826826+0000","last_unstale":"2023-11-28T11:52:50.235900+0000","last_undegraded":"2023-11-28T11:52:50.235900+0000","last_fullsized":"2023-11-28T11:52:50.235900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:23.759602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505901+0000","last_change":"2023-11-28T11:52:50.220598+0000","last_active":"2023-11-28T11:52:51.505901+0000","last_peered":"2023-11-28T11:52:51.505901+0000","last_clean":"2023-11-28T11:52:51.505901+0000","last_became_active":"2023-11-28T11:52:50.219262+0000","last_became_peered":"2023-11-28T11:52:50.219262+0000","last_unstale":"2023-11-28T11:52:51.505901+0000","last_undegraded":"2023-11-28T11:52:51.505901+0000","last_fullsized":"2023-11-28T11:52:51.505901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:07.381326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.234658+0000","last_change":"2023-11-28T11:52:49.821157+0000","last_active":"2023-11-28T11:52:50.234658+0000","last_peered":"2023-11-28T11:52:50.234658+0000","last_clean":"2023-11-28T11:52:50.234658+0000","last_became_active":"2023-11-28T11:52:49.820872+0000","last_became_peered":"2023-11-28T11:52:49.820872+0000","last_unstale":"2023-11-28T11:52:50.234658+0000","last_undegraded":"2023-11-28T11:52:50.234658+0000","last_fullsized":"2023-11-28T11:52:50.234658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221308+0000","last_change":"2023-11-28T11:52:50.229291+0000","last_active":"2023-11-28T11:52:51.221308+0000","last_peered":"2023-11-28T11:52:51.221308+0000","last_clean":"2023-11-28T11:52:51.221308+0000","last_became_active":"2023-11-28T11:52:50.228986+0000","last_became_peered":"2023-11-28T11:52:50.228986+0000","last_unstale":"2023-11-28T11:52:51.221308+0000","last_undegraded":"2023-11-28T11:52:51.221308+0000","last_fullsized":"2023-11-28T11:52:51.221308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:47:21.958196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.217164+0000","last_change":"2023-11-28T11:52:49.823570+0000","last_active":"2023-11-28T11:52:50.217164+0000","last_peered":"2023-11-28T11:52:50.217164+0000","last_clean":"2023-11-28T11:52:50.217164+0000","last_became_active":"2023-11-28T11:52:49.823280+0000","last_became_peered":"2023-11-28T11:52:49.823280+0000","last_unstale":"2023-11-28T11:52:50.217164+0000","last_undegraded":"2023-11-28T11:52:50.217164+0000","last_fullsized":"2023-11-28T11:52:50.217164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:22.861924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505916+0000","last_change":"2023-11-28T11:52:50.231489+0000","last_active":"2023-11-28T11:52:51.505916+0000","last_peered":"2023-11-28T11:52:51.505916+0000","last_clean":"2023-11-28T11:52:51.505916+0000","last_became_active":"2023-11-28T11:52:50.230815+0000","last_became_peered":"2023-11-28T11:52:50.230815+0000","last_unstale":"2023-11-28T11:52:51.505916+0000","last_undegraded":"2023-11-28T11:52:51.505916+0000","last_fullsized":"2023-11-28T11:52:51.505916+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:23.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.218231+0000","last_change":"2023-11-28T11:52:49.822322+0000","last_active":"2023-11-28T11:52:50.218231+0000","last_peered":"2023-11-28T11:52:50.218231+0000","last_clean":"2023-11-28T11:52:50.218231+0000","last_became_active":"2023-11-28T11:52:49.822077+0000","last_became_peered":"2023-11-28T11:52:49.822077+0000","last_unstale":"2023-11-28T11:52:50.218231+0000","last_undegraded":"2023-11-28T11:52:50.218231+0000","last_fullsized":"2023-11-28T11:52:50.218231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:24.019745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504798+0000","last_change":"2023-11-28T11:52:50.229762+0000","last_active":"2023-11-28T11:52:51.504798+0000","last_peered":"2023-11-28T11:52:51.504798+0000","last_clean":"2023-11-28T11:52:51.504798+0000","last_became_active":"2023-11-28T11:52:50.229478+0000","last_became_peered":"2023-11-28T11:52:50.229478+0000","last_unstale":"2023-11-28T11:52:51.504798+0000","last_undegraded":"2023-11-28T11:52:51.504798+0000","last_fullsized":"2023-11-28T11:52:51.504798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:58.036929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.212628+0000","last_change":"2023-11-28T11:52:49.822834+0000","last_active":"2023-11-28T11:52:50.212628+0000","last_peered":"2023-11-28T11:52:50.212628+0000","last_clean":"2023-11-28T11:52:50.212628+0000","last_became_active":"2023-11-28T11:52:49.822565+0000","last_became_peered":"2023-11-28T11:52:49.822565+0000","last_unstale":"2023-11-28T11:52:50.212628+0000","last_undegraded":"2023-11-28T11:52:50.212628+0000","last_fullsized":"2023-11-28T11:52:50.212628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:50.368503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505473+0000","last_change":"2023-11-28T11:52:50.224982+0000","last_active":"2023-11-28T11:52:51.505473+0000","last_peered":"2023-11-28T11:52:51.505473+0000","last_clean":"2023-11-28T11:52:51.505473+0000","last_became_active":"2023-11-28T11:52:50.224664+0000","last_became_peered":"2023-11-28T11:52:50.224664+0000","last_unstale":"2023-11-28T11:52:51.505473+0000","last_undegraded":"2023-11-28T11:52:51.505473+0000","last_fullsized":"2023-11-28T11:52:51.505473+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:35.432520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.215966+0000","last_change":"2023-11-28T11:52:49.821291+0000","last_active":"2023-11-28T11:52:50.215966+0000","last_peered":"2023-11-28T11:52:50.215966+0000","last_clean":"2023-11-28T11:52:50.215966+0000","last_became_active":"2023-11-28T11:52:49.820859+0000","last_became_peered":"2023-11-28T11:52:49.820859+0000","last_unstale":"2023-11-28T11:52:50.215966+0000","last_undegraded":"2023-11-28T11:52:50.215966+0000","last_fullsized":"2023-11-28T11:52:50.215966+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:22.526251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.239975+0000","last_change":"2023-11-28T11:52:50.235221+0000","last_active":"2023-11-28T11:52:50.239975+0000","last_peered":"2023-11-28T11:52:50.239975+0000","last_clean":"2023-11-28T11:52:50.239975+0000","last_became_active":"2023-11-28T11:52:50.234931+0000","last_became_peered":"2023-11-28T11:52:50.234931+0000","last_unstale":"2023-11-28T11:52:50.239975+0000","last_undegraded":"2023-11-28T11:52:50.239975+0000","last_fullsized":"2023-11-28T11:52:50.239975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:30.900577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"45'3","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.233634+0000","last_change":"2023-11-28T11:52:49.826151+0000","last_active":"2023-11-28T11:52:50.233634+0000","last_peered":"2023-11-28T11:52:50.233634+0000","last_clean":"2023-11-28T11:52:50.233634+0000","last_became_active":"2023-11-28T11:52:49.825816+0000","last_became_peered":"2023-11-28T11:52:49.825816+0000","last_unstale":"2023-11-28T11:52:50.233634+0000","last_undegraded":"2023-11-28T11:52:50.233634+0000","last_fullsized":"2023-11-28T11:52:50.233634+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:46.875258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505320+0000","last_change":"2023-11-28T11:52:50.227365+0000","last_active":"2023-11-28T11:52:51.505320+0000","last_peered":"2023-11-28T11:52:51.505320+0000","last_clean":"2023-11-28T11:52:51.505320+0000","last_became_active":"2023-11-28T11:52:50.227092+0000","last_became_peered":"2023-11-28T11:52:50.227092+0000","last_unstale":"2023-11-28T11:52:51.505320+0000","last_undegraded":"2023-11-28T11:52:51.505320+0000","last_fullsized":"2023-11-28T11:52:51.505320+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:17.791185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"45'6","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.242755+0000","last_change":"2023-11-28T11:52:49.825664+0000","last_active":"2023-11-28T11:52:50.242755+0000","last_peered":"2023-11-28T11:52:50.242755+0000","last_clean":"2023-11-28T11:52:50.242755+0000","last_became_active":"2023-11-28T11:52:49.825129+0000","last_became_peered":"2023-11-28T11:52:49.825129+0000","last_unstale":"2023-11-28T11:52:50.242755+0000","last_undegraded":"2023-11-28T11:52:50.242755+0000","last_fullsized":"2023-11-28T11:52:50.242755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:50:16.135396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504959+0000","last_change":"2023-11-28T11:52:50.230323+0000","last_active":"2023-11-28T11:52:51.504959+0000","last_peered":"2023-11-28T11:52:51.504959+0000","last_clean":"2023-11-28T11:52:51.504959+0000","last_became_active":"2023-11-28T11:52:50.230043+0000","last_became_peered":"2023-11-28T11:52:50.230043+0000","last_unstale":"2023-11-28T11:52:51.504959+0000","last_undegraded":"2023-11-28T11:52:51.504959+0000","last_fullsized":"2023-11-28T11:52:51.504959+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:31.723326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.232257+0000","last_change":"2023-11-28T11:52:49.830260+0000","last_active":"2023-11-28T11:52:50.232257+0000","last_peered":"2023-11-28T11:52:50.232257+0000","last_clean":"2023-11-28T11:52:50.232257+0000","last_became_active":"2023-11-28T11:52:49.829977+0000","last_became_peered":"2023-11-28T11:52:49.829977+0000","last_unstale":"2023-11-28T11:52:50.232257+0000","last_undegraded":"2023-11-28T11:52:50.232257+0000","last_fullsized":"2023-11-28T11:52:50.232257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:56:19.733169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.231760+0000","last_change":"2023-11-28T11:52:50.226910+0000","last_active":"2023-11-28T11:52:50.231760+0000","last_peered":"2023-11-28T11:52:50.231760+0000","last_clean":"2023-11-28T11:52:50.231760+0000","last_became_active":"2023-11-28T11:52:50.226630+0000","last_became_peered":"2023-11-28T11:52:50.226630+0000","last_unstale":"2023-11-28T11:52:50.231760+0000","last_undegraded":"2023-11-28T11:52:50.231760+0000","last_fullsized":"2023-11-28T11:52:50.231760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:03.188686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.217545+0000","last_change":"2023-11-28T11:52:49.825164+0000","last_active":"2023-11-28T11:52:50.217545+0000","last_peered":"2023-11-28T11:52:50.217545+0000","last_clean":"2023-11-28T11:52:50.217545+0000","last_became_active":"2023-11-28T11:52:49.824986+0000","last_became_peered":"2023-11-28T11:52:49.824986+0000","last_unstale":"2023-11-28T11:52:50.217545+0000","last_undegraded":"2023-11-28T11:52:50.217545+0000","last_fullsized":"2023-11-28T11:52:50.217545+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221221+0000","last_change":"2023-11-28T11:52:50.223905+0000","last_active":"2023-11-28T11:52:51.221221+0000","last_peered":"2023-11-28T11:52:51.221221+0000","last_clean":"2023-11-28T11:52:51.221221+0000","last_became_active":"2023-11-28T11:52:50.223667+0000","last_became_peered":"2023-11-28T11:52:50.223667+0000","last_unstale":"2023-11-28T11:52:51.221221+0000","last_undegraded":"2023-11-28T11:52:51.221221+0000","last_fullsized":"2023-11-28T11:52:51.221221+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:27:59.207428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.211895+0000","last_change":"2023-11-28T11:52:49.827756+0000","last_active":"2023-11-28T11:52:50.211895+0000","last_peered":"2023-11-28T11:52:50.211895+0000","last_clean":"2023-11-28T11:52:50.211895+0000","last_became_active":"2023-11-28T11:52:49.827476+0000","last_became_peered":"2023-11-28T11:52:49.827476+0000","last_unstale":"2023-11-28T11:52:50.211895+0000","last_undegraded":"2023-11-28T11:52:50.211895+0000","last_fullsized":"2023-11-28T11:52:50.211895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.978385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.232760+0000","last_change":"2023-11-28T11:52:50.228243+0000","last_active":"2023-11-28T11:52:50.232760+0000","last_peered":"2023-11-28T11:52:50.232760+0000","last_clean":"2023-11-28T11:52:50.232760+0000","last_became_active":"2023-11-28T11:52:50.228019+0000","last_became_peered":"2023-11-28T11:52:50.228019+0000","last_unstale":"2023-11-28T11:52:50.232760+0000","last_undegraded":"2023-11-28T11:52:50.232760+0000","last_fullsized":"2023-11-28T11:52:50.232760+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:30.533622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"45'8","reported_seq":19,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.239898+0000","last_change":"2023-11-28T11:52:49.825526+0000","last_active":"2023-11-28T11:52:50.239898+0000","last_peered":"2023-11-28T11:52:50.239898+0000","last_clean":"2023-11-28T11:52:50.239898+0000","last_became_active":"2023-11-28T11:52:49.825217+0000","last_became_peered":"2023-11-28T11:52:49.825217+0000","last_unstale":"2023-11-28T11:52:50.239898+0000","last_undegraded":"2023-11-28T11:52:50.239898+0000","last_fullsized":"2023-11-28T11:52:50.239898+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:44.118206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221046+0000","last_change":"2023-11-28T11:52:50.224840+0000","last_active":"2023-11-28T11:52:51.221046+0000","last_peered":"2023-11-28T11:52:51.221046+0000","last_clean":"2023-11-28T11:52:51.221046+0000","last_became_active":"2023-11-28T11:52:50.224591+0000","last_became_peered":"2023-11-28T11:52:50.224591+0000","last_unstale":"2023-11-28T11:52:51.221046+0000","last_undegraded":"2023-11-28T11:52:51.221046+0000","last_fullsized":"2023-11-28T11:52:51.221046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:33:57.260656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.240132+0000","last_change":"2023-11-28T11:52:49.826554+0000","last_active":"2023-11-28T11:52:50.240132+0000","last_peered":"2023-11-28T11:52:50.240132+0000","last_clean":"2023-11-28T11:52:50.240132+0000","last_became_active":"2023-11-28T11:52:49.826299+0000","last_became_peered":"2023-11-28T11:52:49.826299+0000","last_unstale":"2023-11-28T11:52:50.240132+0000","last_undegraded":"2023-11-28T11:52:50.240132+0000","last_fullsized":"2023-11-28T11:52:50.240132+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:38:33.111523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504152+0000","last_change":"2023-11-28T11:52:50.222350+0000","last_active":"2023-11-28T11:52:51.504152+0000","last_peered":"2023-11-28T11:52:51.504152+0000","last_clean":"2023-11-28T11:52:51.504152+0000","last_became_active":"2023-11-28T11:52:50.222106+0000","last_became_peered":"2023-11-28T11:52:50.222106+0000","last_unstale":"2023-11-28T11:52:51.504152+0000","last_undegraded":"2023-11-28T11:52:51.504152+0000","last_fullsized":"2023-11-28T11:52:51.504152+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:34.989229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.231714+0000","last_change":"2023-11-28T11:52:49.821760+0000","last_active":"2023-11-28T11:52:50.231714+0000","last_peered":"2023-11-28T11:52:50.231714+0000","last_clean":"2023-11-28T11:52:50.231714+0000","last_became_active":"2023-11-28T11:52:49.821354+0000","last_became_peered":"2023-11-28T11:52:49.821354+0000","last_unstale":"2023-11-28T11:52:50.231714+0000","last_undegraded":"2023-11-28T11:52:50.231714+0000","last_fullsized":"2023-11-28T11:52:50.231714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:51.984789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505118+0000","last_change":"2023-11-28T11:52:50.228135+0000","last_active":"2023-11-28T11:52:51.505118+0000","last_peered":"2023-11-28T11:52:51.505118+0000","last_clean":"2023-11-28T11:52:51.505118+0000","last_became_active":"2023-11-28T11:52:50.227849+0000","last_became_peered":"2023-11-28T11:52:50.227849+0000","last_unstale":"2023-11-28T11:52:51.505118+0000","last_undegraded":"2023-11-28T11:52:51.505118+0000","last_fullsized":"2023-11-28T11:52:51.505118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:23.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"45'1","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.237011+0000","last_change":"2023-11-28T11:52:49.826201+0000","last_active":"2023-11-28T11:52:50.237011+0000","last_peered":"2023-11-28T11:52:50.237011+0000","last_clean":"2023-11-28T11:52:50.237011+0000","last_became_active":"2023-11-28T11:52:49.825865+0000","last_became_peered":"2023-11-28T11:52:49.825865+0000","last_unstale":"2023-11-28T11:52:50.237011+0000","last_undegraded":"2023-11-28T11:52:50.237011+0000","last_fullsized":"2023-11-28T11:52:50.237011+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:16:48.119635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504280+0000","last_change":"2023-11-28T11:52:50.223856+0000","last_active":"2023-11-28T11:52:51.504280+0000","last_peered":"2023-11-28T11:52:51.504280+0000","last_clean":"2023-11-28T11:52:51.504280+0000","last_became_active":"2023-11-28T11:52:50.223606+0000","last_became_peered":"2023-11-28T11:52:50.223606+0000","last_unstale":"2023-11-28T11:52:51.504280+0000","last_undegraded":"2023-11-28T11:52:51.504280+0000","last_fullsized":"2023-11-28T11:52:51.504280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:03.982356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.226421+0000","last_change":"2023-11-28T11:52:49.825546+0000","last_active":"2023-11-28T11:52:50.226421+0000","last_peered":"2023-11-28T11:52:50.226421+0000","last_clean":"2023-11-28T11:52:50.226421+0000","last_became_active":"2023-11-28T11:52:49.825206+0000","last_became_peered":"2023-11-28T11:52:49.825206+0000","last_unstale":"2023-11-28T11:52:50.226421+0000","last_undegraded":"2023-11-28T11:52:50.226421+0000","last_fullsized":"2023-11-28T11:52:50.226421+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:23.803406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505181+0000","last_change":"2023-11-28T11:52:50.224313+0000","last_active":"2023-11-28T11:52:51.505181+0000","last_peered":"2023-11-28T11:52:51.505181+0000","last_clean":"2023-11-28T11:52:51.505181+0000","last_became_active":"2023-11-28T11:52:50.224063+0000","last_became_peered":"2023-11-28T11:52:50.224063+0000","last_unstale":"2023-11-28T11:52:51.505181+0000","last_undegraded":"2023-11-28T11:52:51.505181+0000","last_fullsized":"2023-11-28T11:52:51.505181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:58.036929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.230534+0000","last_change":"2023-11-28T11:52:49.822024+0000","last_active":"2023-11-28T11:52:50.230534+0000","last_peered":"2023-11-28T11:52:50.230534+0000","last_clean":"2023-11-28T11:52:50.230534+0000","last_became_active":"2023-11-28T11:52:49.821773+0000","last_became_peered":"2023-11-28T11:52:49.821773+0000","last_unstale":"2023-11-28T11:52:50.230534+0000","last_undegraded":"2023-11-28T11:52:50.230534+0000","last_fullsized":"2023-11-28T11:52:50.230534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:10:56.810434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.220189+0000","last_change":"2023-11-28T11:52:50.226903+0000","last_active":"2023-11-28T11:52:51.220189+0000","last_peered":"2023-11-28T11:52:51.220189+0000","last_clean":"2023-11-28T11:52:51.220189+0000","last_became_active":"2023-11-28T11:52:50.226323+0000","last_became_peered":"2023-11-28T11:52:50.226323+0000","last_unstale":"2023-11-28T11:52:51.220189+0000","last_undegraded":"2023-11-28T11:52:51.220189+0000","last_fullsized":"2023-11-28T11:52:51.220189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:31.880630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.211904+0000","last_change":"2023-11-28T11:52:49.816073+0000","last_active":"2023-11-28T11:52:50.211904+0000","last_peered":"2023-11-28T11:52:50.211904+0000","last_clean":"2023-11-28T11:52:50.211904+0000","last_became_active":"2023-11-28T11:52:49.815798+0000","last_became_peered":"2023-11-28T11:52:49.815798+0000","last_unstale":"2023-11-28T11:52:50.211904+0000","last_undegraded":"2023-11-28T11:52:50.211904+0000","last_fullsized":"2023-11-28T11:52:50.211904+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:54.122734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.232629+0000","last_change":"2023-11-28T11:52:50.227834+0000","last_active":"2023-11-28T11:52:50.232629+0000","last_peered":"2023-11-28T11:52:50.232629+0000","last_clean":"2023-11-28T11:52:50.232629+0000","last_became_active":"2023-11-28T11:52:50.227581+0000","last_became_peered":"2023-11-28T11:52:50.227581+0000","last_unstale":"2023-11-28T11:52:50.232629+0000","last_undegraded":"2023-11-28T11:52:50.232629+0000","last_fullsized":"2023-11-28T11:52:50.232629+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:55:26.868306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.220874+0000","last_change":"2023-11-28T11:52:49.829671+0000","last_active":"2023-11-28T11:52:50.220874+0000","last_peered":"2023-11-28T11:52:50.220874+0000","last_clean":"2023-11-28T11:52:50.220874+0000","last_became_active":"2023-11-28T11:52:49.829404+0000","last_became_peered":"2023-11-28T11:52:49.829404+0000","last_unstale":"2023-11-28T11:52:50.220874+0000","last_undegraded":"2023-11-28T11:52:50.220874+0000","last_fullsized":"2023-11-28T11:52:50.220874+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:51:18.169692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505065+0000","last_change":"2023-11-28T11:52:50.223132+0000","last_active":"2023-11-28T11:52:51.505065+0000","last_peered":"2023-11-28T11:52:51.505065+0000","last_clean":"2023-11-28T11:52:51.505065+0000","last_became_active":"2023-11-28T11:52:50.222845+0000","last_became_peered":"2023-11-28T11:52:50.222845+0000","last_unstale":"2023-11-28T11:52:51.505065+0000","last_undegraded":"2023-11-28T11:52:51.505065+0000","last_fullsized":"2023-11-28T11:52:51.505065+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:14.502256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.214427+0000","last_change":"2023-11-28T11:52:49.823720+0000","last_active":"2023-11-28T11:52:50.214427+0000","last_peered":"2023-11-28T11:52:50.214427+0000","last_clean":"2023-11-28T11:52:50.214427+0000","last_became_active":"2023-11-28T11:52:49.823468+0000","last_became_peered":"2023-11-28T11:52:49.823468+0000","last_unstale":"2023-11-28T11:52:50.214427+0000","last_undegraded":"2023-11-28T11:52:50.214427+0000","last_fullsized":"2023-11-28T11:52:50.214427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:57.828731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.228447+0000","last_change":"2023-11-28T11:52:50.224314+0000","last_active":"2023-11-28T11:52:50.228447+0000","last_peered":"2023-11-28T11:52:50.228447+0000","last_clean":"2023-11-28T11:52:50.228447+0000","last_became_active":"2023-11-28T11:52:50.224067+0000","last_became_peered":"2023-11-28T11:52:50.224067+0000","last_unstale":"2023-11-28T11:52:50.228447+0000","last_undegraded":"2023-11-28T11:52:50.228447+0000","last_fullsized":"2023-11-28T11:52:50.228447+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:04.510511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.212549+0000","last_change":"2023-11-28T11:52:49.825382+0000","last_active":"2023-11-28T11:52:50.212549+0000","last_peered":"2023-11-28T11:52:50.212549+0000","last_clean":"2023-11-28T11:52:50.212549+0000","last_became_active":"2023-11-28T11:52:49.825096+0000","last_became_peered":"2023-11-28T11:52:49.825096+0000","last_unstale":"2023-11-28T11:52:50.212549+0000","last_undegraded":"2023-11-28T11:52:50.212549+0000","last_fullsized":"2023-11-28T11:52:50.212549+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:39.463589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.216540+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:52:50.216540+0000","last_peered":"2023-11-28T11:52:50.216540+0000","last_clean":"2023-11-28T11:52:50.216540+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:52:50.216540+0000","last_undegraded":"2023-11-28T11:52:50.216540+0000","last_fullsized":"2023-11-28T11:52:50.216540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505338+0000","last_change":"2023-11-28T11:52:50.232214+0000","last_active":"2023-11-28T11:52:51.505338+0000","last_peered":"2023-11-28T11:52:51.505338+0000","last_clean":"2023-11-28T11:52:51.505338+0000","last_became_active":"2023-11-28T11:52:50.231965+0000","last_became_peered":"2023-11-28T11:52:50.231965+0000","last_unstale":"2023-11-28T11:52:51.505338+0000","last_undegraded":"2023-11-28T11:52:51.505338+0000","last_fullsized":"2023-11-28T11:52:51.505338+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:07.381326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.212001+0000","last_change":"2023-11-28T11:52:49.826394+0000","last_active":"2023-11-28T11:52:50.212001+0000","last_peered":"2023-11-28T11:52:50.212001+0000","last_clean":"2023-11-28T11:52:50.212001+0000","last_became_active":"2023-11-28T11:52:49.826145+0000","last_became_peered":"2023-11-28T11:52:49.826145+0000","last_unstale":"2023-11-28T11:52:50.212001+0000","last_undegraded":"2023-11-28T11:52:50.212001+0000","last_fullsized":"2023-11-28T11:52:50.212001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:33.980547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":39,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.211874+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:52:50.211874+0000","last_peered":"2023-11-28T11:52:50.211874+0000","last_clean":"2023-11-28T11:52:50.211874+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:52:50.211874+0000","last_undegraded":"2023-11-28T11:52:50.211874+0000","last_fullsized":"2023-11-28T11:52:50.211874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504323+0000","last_change":"2023-11-28T11:52:50.232490+0000","last_active":"2023-11-28T11:52:51.504323+0000","last_peered":"2023-11-28T11:52:51.504323+0000","last_clean":"2023-11-28T11:52:51.504323+0000","last_became_active":"2023-11-28T11:52:50.232236+0000","last_became_peered":"2023-11-28T11:52:50.232236+0000","last_unstale":"2023-11-28T11:52:51.504323+0000","last_undegraded":"2023-11-28T11:52:51.504323+0000","last_fullsized":"2023-11-28T11:52:51.504323+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:37:45.126888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.213352+0000","last_change":"2023-11-28T11:52:49.826946+0000","last_active":"2023-11-28T11:52:50.213352+0000","last_peered":"2023-11-28T11:52:50.213352+0000","last_clean":"2023-11-28T11:52:50.213352+0000","last_became_active":"2023-11-28T11:52:49.826675+0000","last_became_peered":"2023-11-28T11:52:49.826675+0000","last_unstale":"2023-11-28T11:52:50.213352+0000","last_undegraded":"2023-11-28T11:52:50.213352+0000","last_fullsized":"2023-11-28T11:52:50.213352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:34.991301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.215779+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:52:50.215779+0000","last_peered":"2023-11-28T11:52:50.215779+0000","last_clean":"2023-11-28T11:52:50.215779+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:52:50.215779+0000","last_undegraded":"2023-11-28T11:52:50.215779+0000","last_fullsized":"2023-11-28T11:52:50.215779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":184,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.212748+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:52:50.212748+0000","last_peered":"2023-11-28T11:52:50.212748+0000","last_clean":"2023-11-28T11:52:50.212748+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:52:50.212748+0000","last_undegraded":"2023-11-28T11:52:50.212748+0000","last_fullsized":"2023-11-28T11:52:50.212748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221214+0000","last_change":"2023-11-28T11:52:50.232101+0000","last_active":"2023-11-28T11:52:51.221214+0000","last_peered":"2023-11-28T11:52:51.221214+0000","last_clean":"2023-11-28T11:52:51.221214+0000","last_became_active":"2023-11-28T11:52:50.231150+0000","last_became_peered":"2023-11-28T11:52:50.231150+0000","last_unstale":"2023-11-28T11:52:51.221214+0000","last_undegraded":"2023-11-28T11:52:51.221214+0000","last_fullsized":"2023-11-28T11:52:51.221214+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:32.025077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.220219+0000","last_change":"2023-11-28T11:52:49.824383+0000","last_active":"2023-11-28T11:52:50.220219+0000","last_peered":"2023-11-28T11:52:50.220219+0000","last_clean":"2023-11-28T11:52:50.220219+0000","last_became_active":"2023-11-28T11:52:49.824081+0000","last_became_peered":"2023-11-28T11:52:49.824081+0000","last_unstale":"2023-11-28T11:52:50.220219+0000","last_undegraded":"2023-11-28T11:52:50.220219+0000","last_fullsized":"2023-11-28T11:52:50.220219+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:17.464586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.214111+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:52:50.214111+0000","last_peered":"2023-11-28T11:52:50.214111+0000","last_clean":"2023-11-28T11:52:50.214111+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:52:50.214111+0000","last_undegraded":"2023-11-28T11:52:50.214111+0000","last_fullsized":"2023-11-28T11:52:50.214111+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.220042+0000","last_change":"2023-11-28T11:52:50.223647+0000","last_active":"2023-11-28T11:52:51.220042+0000","last_peered":"2023-11-28T11:52:51.220042+0000","last_clean":"2023-11-28T11:52:51.220042+0000","last_became_active":"2023-11-28T11:52:50.223375+0000","last_became_peered":"2023-11-28T11:52:50.223375+0000","last_unstale":"2023-11-28T11:52:51.220042+0000","last_undegraded":"2023-11-28T11:52:51.220042+0000","last_fullsized":"2023-11-28T11:52:51.220042+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:35.999983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"46'1","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.504839+0000","last_change":"2023-11-28T11:52:49.830269+0000","last_active":"2023-11-28T11:52:51.504839+0000","last_peered":"2023-11-28T11:52:51.504839+0000","last_clean":"2023-11-28T11:52:51.504839+0000","last_became_active":"2023-11-28T11:52:49.830011+0000","last_became_peered":"2023-11-28T11:52:49.830011+0000","last_unstale":"2023-11-28T11:52:51.504839+0000","last_undegraded":"2023-11-28T11:52:51.504839+0000","last_fullsized":"2023-11-28T11:52:51.504839+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:05.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.211776+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:52:50.211776+0000","last_peered":"2023-11-28T11:52:50.211776+0000","last_clean":"2023-11-28T11:52:50.211776+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:52:50.211776+0000","last_undegraded":"2023-11-28T11:52:50.211776+0000","last_fullsized":"2023-11-28T11:52:50.211776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.505183+0000","last_change":"2023-11-28T11:52:50.229502+0000","last_active":"2023-11-28T11:52:51.505183+0000","last_peered":"2023-11-28T11:52:51.505183+0000","last_clean":"2023-11-28T11:52:51.505183+0000","last_became_active":"2023-11-28T11:52:50.229181+0000","last_became_peered":"2023-11-28T11:52:50.229181+0000","last_unstale":"2023-11-28T11:52:51.505183+0000","last_undegraded":"2023-11-28T11:52:51.505183+0000","last_fullsized":"2023-11-28T11:52:51.505183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:17.791185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"45'2","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.237078+0000","last_change":"2023-11-28T11:52:49.824449+0000","last_active":"2023-11-28T11:52:50.237078+0000","last_peered":"2023-11-28T11:52:50.237078+0000","last_clean":"2023-11-28T11:52:50.237078+0000","last_became_active":"2023-11-28T11:52:49.824190+0000","last_became_peered":"2023-11-28T11:52:49.824190+0000","last_unstale":"2023-11-28T11:52:50.237078+0000","last_undegraded":"2023-11-28T11:52:50.237078+0000","last_fullsized":"2023-11-28T11:52:50.237078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:38.386176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.219438+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:52:50.219438+0000","last_peered":"2023-11-28T11:52:50.219438+0000","last_clean":"2023-11-28T11:52:50.219438+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:52:50.219438+0000","last_undegraded":"2023-11-28T11:52:50.219438+0000","last_fullsized":"2023-11-28T11:52:50.219438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-28T11:52:51.221378+0000","last_change":"2023-11-28T11:52:50.230867+0000","last_active":"2023-11-28T11:52:51.221378+0000","last_peered":"2023-11-28T11:52:51.221378+0000","last_clean":"2023-11-28T11:52:51.221378+0000","last_became_active":"2023-11-28T11:52:50.229473+0000","last_became_peered":"2023-11-28T11:52:50.229473+0000","last_unstale":"2023-11-28T11:52:51.221378+0000","last_undegraded":"2023-11-28T11:52:51.221378+0000","last_fullsized":"2023-11-28T11:52:51.221378+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:52:56.274311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.219405+0000","last_change":"2023-11-28T11:52:49.822982+0000","last_active":"2023-11-28T11:52:50.219405+0000","last_peered":"2023-11-28T11:52:50.219405+0000","last_clean":"2023-11-28T11:52:50.219405+0000","last_became_active":"2023-11-28T11:52:49.822363+0000","last_became_peered":"2023-11-28T11:52:49.822363+0000","last_unstale":"2023-11-28T11:52:50.219405+0000","last_undegraded":"2023-11-28T11:52:50.219405+0000","last_fullsized":"2023-11-28T11:52:50.219405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:30.871948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.216657+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:52:50.216657+0000","last_peered":"2023-11-28T11:52:50.216657+0000","last_clean":"2023-11-28T11:52:50.216657+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:52:50.216657+0000","last_undegraded":"2023-11-28T11:52:50.216657+0000","last_fullsized":"2023-11-28T11:52:50.216657+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":10,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.240111+0000","last_change":"2023-11-28T11:52:50.236369+0000","last_active":"2023-11-28T11:52:50.240111+0000","last_peered":"2023-11-28T11:52:50.240111+0000","last_clean":"2023-11-28T11:52:50.240111+0000","last_became_active":"2023-11-28T11:52:50.236121+0000","last_became_peered":"2023-11-28T11:52:50.236121+0000","last_unstale":"2023-11-28T11:52:50.240111+0000","last_undegraded":"2023-11-28T11:52:50.240111+0000","last_fullsized":"2023-11-28T11:52:50.240111+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:49.804716+0000","last_clean_scrub_stamp":"2023-11-28T11:52:49.804716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:55.616287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.215826+0000","last_change":"2023-11-28T11:52:49.817256+0000","last_active":"2023-11-28T11:52:50.215826+0000","last_peered":"2023-11-28T11:52:50.215826+0000","last_clean":"2023-11-28T11:52:50.215826+0000","last_became_active":"2023-11-28T11:52:49.816922+0000","last_became_peered":"2023-11-28T11:52:49.816922+0000","last_unstale":"2023-11-28T11:52:50.215826+0000","last_undegraded":"2023-11-28T11:52:50.215826+0000","last_fullsized":"2023-11-28T11:52:50.215826+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:52:48.796034+0000","last_clean_scrub_stamp":"2023-11-28T11:52:48.796034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:48:34.060703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":36,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-28T11:52:50.217268+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:52:50.217268+0000","last_peered":"2023-11-28T11:52:50.217268+0000","last_clean":"2023-11-28T11:52:50.217268+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:52:50.217268+0000","last_undegraded":"2023-11-28T11:52:50.217268+0000","last_fullsized":"2023-11-28T11:52:50.217268+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10144,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361696,"statfs":{"total":96636764160,"available":96626376704,"internally_reserved":0,"allocated":1138688,"data_stored":818749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.68000000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55700000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.70999999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64400000000000002}]}]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8796,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363044,"statfs":{"total":96636764160,"available":96627757056,"internally_reserved":0,"allocated":544768,"data_stored":228893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.621}]}]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9952,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94361888,"statfs":{"total":96636764160,"available":96626573312,"internally_reserved":0,"allocated":1138688,"data_stored":818785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.27300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44}]},{"osd":5,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.433},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.44700000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45500000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:52:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42199999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8920,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362920,"statfs":{"total":96636764160,"available":96627630080,"internally_reserved":0,"allocated":540672,"data_stored":227327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.17299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"last":0.34399999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"last":0.371},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.40000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:52:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.67000000000000004}]}]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8800,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363040,"statfs":{"total":96636764160,"available":96627752960,"internally_reserved":0,"allocated":548864,"data_stored":229805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.317},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":8.2970000000000006,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.28399999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.4039999999999999,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.3440000000000003,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":8.359,"5min":8.359,"15min":8.359},"last":0.55500000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.3810000000000002,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.439},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48699999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":8.3339999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6820000000000004,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.45300000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9124,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362716,"statfs":{"total":96636764160,"available":96627421184,"internally_reserved":0,"allocated":552960,"data_stored":229827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":16.231000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.52200000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:52:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79900000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8984,"kb_used_data":528,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362856,"statfs":{"total":96636764160,"available":96627564544,"internally_reserved":0,"allocated":540672,"data_stored":228823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.432},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.378},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:52:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.501},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53800000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8912,"kb_used_data":520,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362928,"statfs":{"total":96636764160,"available":96627638272,"internally_reserved":0,"allocated":532480,"data_stored":227271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.502},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.628},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60299999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:52:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:52:59.173 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:52:59.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:52:59.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.335+0000 7fad7fd69700 1 Processor -- start 2023-11-28T11:52:59.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.335+0000 7fad7fd69700 1 -- start start 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7fd69700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fad780cd8c0 0x7fad78140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7fd69700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7813ff00 con 0x7fad7806f850 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7fd69700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad781404c0 con 0x7fad780cdc90 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7fd69700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7815aa80 con 0x7fad780cd8c0 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7e306700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fad7806f850 0x7fad7806fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54200/0 (socket says 172.21.15.144:54200) 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7e306700 1 -- 172.21.15.144:0/1355510670 learned_addr learned my addr 172.21.15.144:0/1355510670 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:59.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7db05700 1 --2- 172.21.15.144:0/1355510670 >> v2:172.21.15.177:3300/0 conn(0x7fad780cd8c0 0x7fad78140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549221047 0 0) 0x7fad7813ff00 con 0x7fad7806f850 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad640035b0 con 0x7fad7806f850 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581932503 0 0) 0x7fad781404c0 con 0x7fad780cdc90 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad7813ff00 con 0x7fad780cdc90 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809465231 0 0) 0x7fad640035b0 con 0x7fad7806f850 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 >> v2:172.21.15.177:3300/0 conn(0x7fad780cd8c0 msgr2=0x7fad78140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 --2- 172.21.15.144:0/1355510670 >> v2:172.21.15.177:3300/0 conn(0x7fad780cd8c0 0x7fad78140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 >> v1:172.21.15.177:6789/0 conn(0x7fad780cdc90 legacy=0x7fad780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780b3040 con 0x7fad7806f850 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.336+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fad700019f0 con 0x7fad7806f850 2023-11-28T11:52:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 >> v1:172.21.15.144:6789/0 conn(0x7fad7806f850 legacy=0x7fad7806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad64003b90 0x7fad64003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad64004850 0x7fad64004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad6400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7815aa80 con 0x7fad64005200 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6400c110 con 0x7fad64004850 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6400c2d0 con 0x7fad64003b90 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad6400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7db05700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad64003b90 0x7fad64003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7d304700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad64004850 0x7fad64004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 -- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad64003b90 msgr2=0x7fad64003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad64003b90 0x7fad64003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 -- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad64004850 msgr2=0x7fad64004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad64004850 0x7fad64004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 -- 172.21.15.144:0/1355510670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640035b0 con 0x7fad64005200 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 -- 172.21.15.144:0/1355510670 reap_dead start 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7e306700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad6400ba10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fad70002230 tx=0x7fad64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83eb9d44b397d629 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700072d0 con 0x7fad64005200 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70007430 con 0x7fad64005200 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.337+0000 7fad7cb03700 1 -- 172.21.15.144:0/1355510670 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700077c0 con 0x7fad64005200 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 -- 172.21.15.144:0/1355510670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 msgr2=0x7fad6400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad6400ba10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fad70002230 tx=0x7fad64005880 comp rx=0 tx=0).stop 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 -- 172.21.15.144:0/1355510670 shutdown_connections 2023-11-28T11:52:59.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 --2- 172.21.15.144:0/1355510670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad6400ba10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 -- 172.21.15.144:0/1355510670 >> 172.21.15.144:0/1355510670 conn(0x7fad7806aed0 msgr2=0x7fad7806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 -- 172.21.15.144:0/1355510670 shutdown_connections 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.338+0000 7fad7fd69700 1 -- 172.21.15.144:0/1355510670 wait complete. 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.339+0000 7fad7fd69700 1 Processor -- start 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.339+0000 7fad7fd69700 1 -- start start 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7fd69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad780b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7fd69700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad78156610 0x7fad780b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7fd69700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad7813e2b0 0x7fad7813e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7fd69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7813ec60 con 0x7fad64005200 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7fd69700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780b68d0 con 0x7fad78156610 2023-11-28T11:52:59.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7fd69700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780b5cc0 con 0x7fad7813e2b0 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad780b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad780b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37496/0 (socket says 172.21.15.144:37496) 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 -- 172.21.15.144:0/22598890 learned_addr learned my addr 172.21.15.144:0/22598890 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7d304700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad78156610 0x7fad780b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7e306700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad7813e2b0 0x7fad7813e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 -- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad7813e2b0 msgr2=0x7fad7813e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad7813e2b0 0x7fad7813e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 -- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad78156610 msgr2=0x7fad780b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad78156610 0x7fad780b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 -- 172.21.15.144:0/22598890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780b35f0 con 0x7fad64005200 2023-11-28T11:52:59.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad7db05700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad780b56e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fad74009fd0 tx=0x7fad74011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:59.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.340+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74009610 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.341+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74005c30 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.341+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400f6d0 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.341+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780b3310 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.341+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad78140150 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.342+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78068690 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.342+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad7400f830 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.342+0000 7fad6e7fc700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad5c0682f0 0x7fad5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.342+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad74086520 con 0x7fad64005200 2023-11-28T11:52:59.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.342+0000 7fad7d304700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad5c0682f0 0x7fad5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.343+0000 7fad7d304700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad5c0682f0 0x7fad5c06a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad780cd560 tx=0x7fad68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:59.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.347+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad74052d80 con 0x7fad64005200 2023-11-28T11:52:59.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.603+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fad780b4fe0 con 0x7fad64005200 2023-11-28T11:52:59.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.607+0000 7fad6e7fc700 1 -- 172.21.15.144:0/22598890 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fad740563a0 con 0x7fad64005200 2023-11-28T11:52:59.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad5c0682f0 msgr2=0x7fad5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad5c0682f0 0x7fad5c06a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad780cd560 tx=0x7fad68009040 comp rx=0 tx=0).stop 2023-11-28T11:52:59.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 msgr2=0x7fad780b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad780b56e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fad74009fd0 tx=0x7fad74011d30 comp rx=0 tx=0).stop 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 shutdown_connections 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad64005200 0x7fad780b56e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fad5c0682f0 0x7fad5c06a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad7813e2b0 0x7fad7813e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 --2- 172.21.15.144:0/22598890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad78156610 0x7fad780b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 >> 172.21.15.144:0/22598890 conn(0x7fad7806aed0 msgr2=0x7fad780b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 shutdown_connections 2023-11-28T11:52:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.609+0000 7fad7fd69700 1 -- 172.21.15.144:0/22598890 wait complete. 2023-11-28T11:52:59.613 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.0 2023-11-28T11:52:59.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:52:59.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.742+0000 7f5d82c09700 1 Processor -- start 2023-11-28T11:52:59.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.742+0000 7f5d82c09700 1 -- start start 2023-11-28T11:52:59.751 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d82c09700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 0x7f5d7c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d82c09700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c15d060 con 0x7f5d7c160540 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d82c09700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c15d340 con 0x7f5d7c071040 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d82c09700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c15d620 con 0x7f5d7c06dee0 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d811a6700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5d7c160540 0x7f5d7c1609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54220/0 (socket says 172.21.15.144:54220) 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d811a6700 1 -- 172.21.15.144:0/2912779614 learned_addr learned my addr 172.21.15.144:0/2912779614 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d7bfff700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 0x7f5d7c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.743+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256610626 0 0) 0x7f5d7c15d060 con 0x7f5d7c160540 2023-11-28T11:52:59.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d6c0035b0 con 0x7f5d7c160540 2023-11-28T11:52:59.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986907773 0 0) 0x7f5d7c15d340 con 0x7f5d7c071040 2023-11-28T11:52:59.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c15d060 con 0x7f5d7c071040 2023-11-28T11:52:59.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139506305 0 0) 0x7f5d6c0035b0 con 0x7f5d7c160540 2023-11-28T11:52:59.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 msgr2=0x7f5d7c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:59.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 0x7f5d7c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 >> v1:172.21.15.177:6789/0 conn(0x7f5d7c071040 legacy=0x7f5d7c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c15d900 con 0x7f5d7c160540 2023-11-28T11:52:59.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f5d74001970 con 0x7f5d7c160540 2023-11-28T11:52:59.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 >> v1:172.21.15.144:6789/0 conn(0x7f5d7c160540 legacy=0x7f5d7c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c003bc0 0x7f5d6c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c004850 0x7f5d6c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 0x7f5d6c005320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c15d620 con 0x7f5d6c003bc0 2023-11-28T11:52:59.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c00bbf0 con 0x7f5d6c004850 2023-11-28T11:52:59.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c00bdb0 con 0x7f5d7c06dee0 2023-11-28T11:52:59.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c003bc0 0x7f5d6c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d7bfff700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c004850 0x7f5d6c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d811a6700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 0x7f5d6c005320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 -- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 msgr2=0x7f5d6c005320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d7c06dee0 0x7f5d6c005320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 -- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c004850 msgr2=0x7f5d6c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c004850 0x7f5d6c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 -- 172.21.15.144:0/2912779614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c0035b0 con 0x7f5d6c003bc0 2023-11-28T11:52:59.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.744+0000 7f5d809a5700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c003bc0 0x7f5d6c003fb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5d70001940 tx=0x7f5d6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5280f7bfd81038ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:59.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70001430 con 0x7f5d6c003bc0 2023-11-28T11:52:59.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70001d70 con 0x7f5d6c003bc0 2023-11-28T11:52:59.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d7b7fe700 1 -- 172.21.15.144:0/2912779614 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70005510 con 0x7f5d6c003bc0 2023-11-28T11:52:59.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d82c09700 1 -- 172.21.15.144:0/2912779614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c003bc0 msgr2=0x7f5d6c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d82c09700 1 --2- 172.21.15.144:0/2912779614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c003bc0 0x7f5d6c003fb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5d70001940 tx=0x7f5d6c005880 comp rx=0 tx=0).stop 2023-11-28T11:52:59.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d811a6700 1 -- 172.21.15.144:0/2912779614 reap_dead start 2023-11-28T11:52:59.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d82c09700 1 -- 172.21.15.144:0/2912779614 shutdown_connections 2023-11-28T11:52:59.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d82c09700 1 -- 172.21.15.144:0/2912779614 >> 172.21.15.144:0/2912779614 conn(0x7f5d7c0d0000 msgr2=0x7f5d7c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:59.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d82c09700 1 -- 172.21.15.144:0/2912779614 shutdown_connections 2023-11-28T11:52:59.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.745+0000 7f5d82c09700 1 -- 172.21.15.144:0/2912779614 wait complete. 2023-11-28T11:52:59.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.746+0000 7f5d82c09700 1 Processor -- start 2023-11-28T11:52:59.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.746+0000 7f5d82c09700 1 -- start start 2023-11-28T11:52:59.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d82c09700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d6c003bc0 0x7f5d7c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d82c09700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d7c0c3a60 0x7f5d7c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d82c09700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 0x7f5d7c15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d82c09700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c158710 con 0x7f5d7c0b5650 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d82c09700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0c3f60 con 0x7f5d7c0c3a60 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d82c09700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0c4240 con 0x7f5d6c003bc0 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d811a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 0x7f5d7c15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d811a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 0x7f5d7c15b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37520/0 (socket says 172.21.15.144:37520) 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d811a6700 1 -- 172.21.15.144:0/3023519483 learned_addr learned my addr 172.21.15.144:0/3023519483 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:52:59.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d809a5700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d6c003bc0 0x7f5d7c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.747+0000 7f5d7bfff700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d7c0c3a60 0x7f5d7c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d811a6700 1 -- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d6c003bc0 msgr2=0x7f5d7c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d811a6700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d6c003bc0 0x7f5d7c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d811a6700 1 -- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d7c0c3a60 msgr2=0x7f5d7c15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d811a6700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d7c0c3a60 0x7f5d7c15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d811a6700 1 -- 172.21.15.144:0/3023519483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c06a270 con 0x7f5d7c0b5650 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d811a6700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 0x7f5d7c15b9d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000f80 tx=0x7f5d7400f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400fbc0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7400fd20 con 0x7f5d7c0b5650 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c15ccd0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74013610 con 0x7f5d7c0b5650 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.748+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c0c4800 con 0x7f5d7c0b5650 2023-11-28T11:52:59.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.749+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c06abd0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.749+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d74013770 con 0x7f5d7c0b5650 2023-11-28T11:52:59.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.750+0000 7f5d797fa700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d60068320 0x7f5d6006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:52:59.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.750+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d740861a0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.750+0000 7f5d809a5700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d60068320 0x7f5d6006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:52:59.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.750+0000 7f5d809a5700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d60068320 0x7f5d6006a7d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d70015040 tx=0x7f5d7000a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:52:59.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.754+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d7c06abd0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.978+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d7c06abd0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.978+0000 7f5d797fa700 1 -- 172.21.15.144:0/3023519483 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f5d7c06abd0 con 0x7f5d7c0b5650 2023-11-28T11:52:59.980 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:52:59.980 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:52:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d60068320 msgr2=0x7f5d6006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d60068320 0x7f5d6006a7d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d70015040 tx=0x7f5d7000a000 comp rx=0 tx=0).stop 2023-11-28T11:52:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 msgr2=0x7f5d7c15b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:52:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 0x7f5d7c15b9d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000f80 tx=0x7f5d7400f4b0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 shutdown_connections 2023-11-28T11:52:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d7c0b5650 0x7f5d7c15b9d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d60068320 0x7f5d6006a7d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d6c003bc0 0x7f5d7c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 --2- 172.21.15.144:0/3023519483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d7c0c3a60 0x7f5d7c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:52:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 >> 172.21.15.144:0/3023519483 conn(0x7f5d7c0d0000 msgr2=0x7f5d7c0aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:52:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.980+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 shutdown_connections 2023-11-28T11:52:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:52:59.981+0000 7f5d82c09700 1 -- 172.21.15.144:0/3023519483 wait complete. 2023-11-28T11:52:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:00.397 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:52:50.207501+0000', 'modified': '2023-11-28T11:52:53.224270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5535': {'gid': 5535, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6834/1698439070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 1698439070}, {'type': 'v1', 'addr': '172.21.15.177:6834', 'nonce': 1698439070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:53:00.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:53:00.398 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:53:00.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.524+0000 7f6825563700 1 Processor -- start 2023-11-28T11:53:00.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.524+0000 7f6825563700 1 -- start start 2023-11-28T11:53:00.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f6825563700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6820159490 0x7f68200b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f6825563700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159860 0x7f68200b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f6825563700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68200ccc00 con 0x7f68200cdc80 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f6825563700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68200ccec0 con 0x7f6820159490 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f6825563700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200cd1a0 con 0x7f6820159860 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159860 0x7f68200b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681effd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6820159490 0x7f68200b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681f7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f68200cdc80 0x7f68200c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:54730/0 (socket says 172.21.15.144:54730) 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681f7fe700 1 -- 172.21.15.144:0/2163981646 learned_addr learned my addr 172.21.15.144:0/2163981646 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:00.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159860 0x7f68200b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:49608/0 (socket says 172.21.15.144:49608) 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681effd700 1 -- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159860 msgr2=0x7f68200b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681effd700 1 --2- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159860 0x7f68200b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681effd700 1 -- 172.21.15.144:0/2163981646 >> v1:172.21.15.177:6789/0 conn(0x7f68200cdc80 legacy=0x7f68200c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.525+0000 7f681effd700 1 -- 172.21.15.144:0/2163981646 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200cd480 con 0x7f6820159490 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f681effd700 1 --2- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6820159490 0x7f68200b0670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6810009840 tx=0x7f6810009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d99e2342d3c4ea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f681dffb700 1 -- 172.21.15.144:0/2163981646 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810011830 con 0x7f6820159490 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f681dffb700 1 -- 172.21.15.144:0/2163981646 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810011990 con 0x7f6820159490 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f681dffb700 1 -- 172.21.15.144:0/2163981646 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810011d20 con 0x7f6820159490 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 -- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6820159490 msgr2=0x7f68200b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 --2- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6820159490 0x7f68200b0670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6810009840 tx=0x7f6810009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:00.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 -- 172.21.15.144:0/2163981646 shutdown_connections 2023-11-28T11:53:00.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 --2- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6820159490 0x7f68200b0670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 --2- 172.21.15.144:0/2163981646 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159860 0x7f68200b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 -- 172.21.15.144:0/2163981646 >> 172.21.15.144:0/2163981646 conn(0x7f68200cb670 msgr2=0x7f68200cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 -- 172.21.15.144:0/2163981646 shutdown_connections 2023-11-28T11:53:00.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.526+0000 7f6825563700 1 -- 172.21.15.144:0/2163981646 wait complete. 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 Processor -- start 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 -- start start 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 0x7f6820155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159490 0x7f6820154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68201e3c40 0x7f6820153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201e4490 con 0x7f68200cdc80 2023-11-28T11:53:00.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201e41b0 con 0x7f68201e3c40 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f6825563700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68200b1bc0 con 0x7f6820159490 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f681effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 0x7f6820155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f681effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 0x7f6820155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37530/0 (socket says 172.21.15.144:37530) 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f681effd700 1 -- 172.21.15.144:0/2826810914 learned_addr learned my addr 172.21.15.144:0/2826810914 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.528+0000 7f681e7fc700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159490 0x7f6820154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681f7fe700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68201e3c40 0x7f6820153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681effd700 1 -- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159490 msgr2=0x7f6820154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681effd700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159490 0x7f6820154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681effd700 1 -- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68201e3c40 msgr2=0x7f6820153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681effd700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68201e3c40 0x7f6820153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681effd700 1 -- 172.21.15.144:0/2826810914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100094a0 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f681effd700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 0x7f6820155e40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6810009fd0 tx=0x7f6810004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f68077fe700 1 -- 172.21.15.144:0/2826810914 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100036a0 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f68077fe700 1 -- 172.21.15.144:0/2826810914 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681001f070 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68200c9bd0 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f68077fe700 1 -- 172.21.15.144:0/2826810914 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810003a60 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.529+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68200cd760 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.531+0000 7f68077fe700 1 -- 172.21.15.144:0/2826810914 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f681000f050 con 0x7f68200cdc80 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.531+0000 7f68077fe700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f680c0682f0 0x7f680c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.531+0000 7f681e7fc700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f680c0682f0 0x7f680c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.531+0000 7f68077fe700 1 -- 172.21.15.144:0/2826810914 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6810021070 con 0x7f68200cdc80 2023-11-28T11:53:00.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.532+0000 7f681e7fc700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f680c0682f0 0x7f680c06a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f682006d920 tx=0x7f6814009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.533+0000 7f68316ecb80 1 Processor -- start 2023-11-28T11:53:00.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.533+0000 7f68316ecb80 1 -- start start 2023-11-28T11:53:00.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247caaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 0x55b247cd4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd5120 0x55b247cf0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f68316ecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247b94bc0 con 0x55b247cd2130 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247c03350 con 0x55b247cd5120 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b247bfc880 con 0x55b247cd4810 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 0x55b247cd4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 0x55b247cd4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37538/0 (socket says 172.21.15.144:37538) 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 -- 172.21.15.144:0/594451007 learned_addr learned my addr 172.21.15.144:0/594451007 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681f7fe700 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd5120 0x55b247cf0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681effd700 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247caaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 -- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 msgr2=0x55b247caaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247caaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 -- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd5120 msgr2=0x55b247cf0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd5120 0x55b247cf0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 -- 172.21.15.144:0/594451007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b247a7e5e0 con 0x55b247cd2130 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681f7fe700 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd5120 0x55b247cf0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.534+0000 7f681e7fc700 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 0x55b247cd4be0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6814004560 tx=0x7f6814014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c5b8d164baf6fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f6804ff9700 1 -- 172.21.15.144:0/594451007 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681401c040 con 0x55b247cd2130 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f6804ff9700 1 -- 172.21.15.144:0/594451007 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814004590 con 0x55b247cd2130 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f6804ff9700 1 -- 172.21.15.144:0/594451007 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814003ab0 con 0x55b247cd2130 2023-11-28T11:53:00.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 -- 172.21.15.144:0/594451007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 msgr2=0x55b247cd4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 0x55b247cd4be0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6814004560 tx=0x7f6814014000 comp rx=0 tx=0).stop 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 -- 172.21.15.144:0/594451007 shutdown_connections 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd5120 0x55b247cf0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247caaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 --2- 172.21.15.144:0/594451007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd2130 0x55b247cd4be0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 -- 172.21.15.144:0/594451007 >> 172.21.15.144:0/594451007 conn(0x55b247ca7d60 msgr2=0x55b247ca86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 -- 172.21.15.144:0/594451007 shutdown_connections 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.535+0000 7f68316ecb80 1 -- 172.21.15.144:0/594451007 wait complete. 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.536+0000 7f68316ecb80 1 -- start start 2023-11-28T11:53:00.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.536+0000 7f68316ecb80 1 Processor -- start 2023-11-28T11:53:00.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247cd7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.550 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247cd8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 0x55b247cf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.551 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247a808d0 con 0x55b247cd5120 2023-11-28T11:53:00.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247ccdf10 con 0x55b247cd2130 2023-11-28T11:53:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b247ce9660 con 0x55b247cd4810 2023-11-28T11:53:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 10 client.0 _ll_register_callbacks cb 0x55b247aa2240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 0x55b247cf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 0x55b247cf4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37548/0 (socket says 172.21.15.144:37548) 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 -- 172.21.15.144:0/2447626751 learned_addr learned my addr 172.21.15.144:0/2447626751 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681effd700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247cd7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 -- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 msgr2=0x55b247cd8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681e7fc700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247cd8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247cd8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 -- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 msgr2=0x55b247cd7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247cd7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 -- 172.21.15.144:0/2447626751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6814004210 con 0x55b247cd5120 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f681f7fe700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 0x55b247cf4ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6818007860 tx=0x7f6818007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ebae8ffa76ee72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f67fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f67fdffb700 1 -- 172.21.15.144:0/2447626751 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818010ae0 con 0x55b247cd5120 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f67fdffb700 1 -- 172.21.15.144:0/2447626751 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6818010c40 con 0x55b247cd5120 2023-11-28T11:53:00.555 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.537+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b247bca140 con 0x55b247cd5120 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f67fdffb700 1 -- 172.21.15.144:0/2447626751 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6818010e00 con 0x55b247cd5120 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 10 client.5691 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b247a2ad90 con 0x55b247cd5120 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f67fdffb700 1 -- 172.21.15.144:0/2447626751 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b247bca140 con 0x55b247cd5120 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f67fdffb700 1 -- 172.21.15.144:0/2447626751 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6818014400 con 0x55b247cd5120 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 10 client.5691 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 10 client.5691 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 20 client.5691 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cf67c0 0x55b247cf5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 4 client.5691 mds_command: new command op to 5535 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b247a80f10 con 0x55b247cf67c0 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f681effd700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cf67c0 0x55b247cf5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.556 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f67fdffb700 10 client.5691 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.538+0000 7f681effd700 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cf67c0 0x55b247cf5380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90ed1ee13b94a67 server_cookie=76ccb865198abe4c in_seq=0 out_seq=0 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f67fdffb700 1 -- 172.21.15.144:0/2447626751 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55b247a80f10 con 0x55b247cf67c0 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f67fdffb700 10 client.5691 handle_command_reply: tid=0 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 client.5691 shutdown 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 msgr2=0x55b247cf4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 0x55b247cf4ab0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6818007860 tx=0x7f6818007c20 comp rx=0 tx=0).stop 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 shutdown_connections 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd5120 0x55b247cf4ab0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cf67c0 0x55b247cf5380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247cd7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 --2- 172.21.15.144:0/2447626751 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd4810 0x55b247cd8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f67fdffb700 0 client.5691 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 >> 172.21.15.144:0/2447626751 conn(0x55b247ca7d60 msgr2=0x55b247caa990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 shutdown_connections 2023-11-28T11:53:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.539+0000 7f68316ecb80 1 -- 172.21.15.144:0/2447626751 wait complete. 2023-11-28T11:53:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.540+0000 7f68316ecb80 20 client.5691 trim_cache size 0 max 16384 2023-11-28T11:53:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 Processor -- start 2023-11-28T11:53:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 -- start start 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247ca5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247ca3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247ca3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247cc0460 con 0x55b247cd4810 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247ce8610 con 0x55b247cd2130 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.557+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b247cc0bd0 con 0x55b247cd5120 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247ca3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247ca3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37554/0 (socket says 172.21.15.144:37554) 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 -- 172.21.15.144:0/3191061465 learned_addr learned my addr 172.21.15.144:0/3191061465 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681f7fe700 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247ca3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 -- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 msgr2=0x55b247ca3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681effd700 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247ca5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247ca3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 -- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 msgr2=0x55b247ca5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247ca5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 -- 172.21.15.144:0/3191061465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b247bc3140 con 0x55b247cd4810 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681effd700 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247ca5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f681e7fc700 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247ca3450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6814004500 tx=0x7f6814016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631b878b3c52fd15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f68057fa700 1 -- 172.21.15.144:0/3191061465 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814003ab0 con 0x55b247cd4810 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f68057fa700 1 -- 172.21.15.144:0/3191061465 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6814031070 con 0x55b247cd4810 2023-11-28T11:53:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.558+0000 7f68057fa700 1 -- 172.21.15.144:0/3191061465 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814003e50 con 0x55b247cd4810 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 -- 172.21.15.144:0/3191061465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 msgr2=0x55b247ca3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247ca3450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6814004500 tx=0x7f6814016000 comp rx=0 tx=0).stop 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 -- 172.21.15.144:0/3191061465 shutdown_connections 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247ca3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247ca5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3191061465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247ca3450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 -- 172.21.15.144:0/3191061465 >> 172.21.15.144:0/3191061465 conn(0x55b247ca7d60 msgr2=0x55b247caba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 -- 172.21.15.144:0/3191061465 shutdown_connections 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 -- 172.21.15.144:0/3191061465 wait complete. 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 -- start start 2023-11-28T11:53:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.559+0000 7f68316ecb80 1 Processor -- start 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247d4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247caa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247caabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247d4f800 con 0x55b247cd4810 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b247cc8790 con 0x55b247cd2130 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b247d08280 con 0x55b247cd5120 2023-11-28T11:53:00.568 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f68316ecb80 10 client.0 _ll_register_callbacks cb 0x7f68180073f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247caa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247caa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37560/0 (socket says 172.21.15.144:37560) 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 -- 172.21.15.144:0/3971613223 learned_addr learned my addr 172.21.15.144:0/3971613223 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681effd700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247d4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681f7fe700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247caabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 -- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 msgr2=0x55b247caabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247caabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 -- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 msgr2=0x55b247d4e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247d4e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.560+0000 7f681e7fc700 1 -- 172.21.15.144:0/3971613223 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b247a7e5e0 con 0x55b247cd4810 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f681e7fc700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247caa690 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6814023000 tx=0x7f68140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aefcdf0ffb64522 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f67fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:00.569 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f67fdffb700 1 -- 172.21.15.144:0/3971613223 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814003c40 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f67fdffb700 1 -- 172.21.15.144:0/3971613223 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6814003da0 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b247bba160 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f67fdffb700 1 -- 172.21.15.144:0/3971613223 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6814022470 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 10 client.5697 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b247a51200 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f67fdffb700 1 -- 172.21.15.144:0/3971613223 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b247bba160 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f67fdffb700 1 -- 172.21.15.144:0/3971613223 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f68140225d0 con 0x55b247cd4810 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 10 client.5697 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 10 client.5697 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 20 client.5697 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cc2e20 0x55b247cc31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 4 client.5697 mds_command: new command op to 5535 tid=0[{"prefix": "status"}] 2023-11-28T11:53:00.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b247ba4170 con 0x55b247cc2e20 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.561+0000 7f681effd700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cc2e20 0x55b247cc31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f67fdffb700 10 client.5697 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f681effd700 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cc2e20 0x55b247cc31f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe05dec36eb445b6 server_cookie=fb2e5d1e55115a1a in_seq=0 out_seq=0 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f67fdffb700 1 -- 172.21.15.144:0/3971613223 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b247ba4170 con 0x55b247cc2e20 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f67fdffb700 10 client.5697 handle_command_reply: tid=0 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 client.5697 shutdown 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 msgr2=0x55b247caa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247caa690 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6814023000 tx=0x7f68140047c0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 shutdown_connections 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b247cc2e20 0x55b247cc31f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b247cd5120 0x55b247caabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f67fdffb700 0 client.5697 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:00.571 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b247cd2130 0x55b247d4e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 --2- 172.21.15.144:0/3971613223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b247cd4810 0x55b247caa690 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 >> 172.21.15.144:0/3971613223 conn(0x55b247ca7d60 msgr2=0x55b247ccbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 shutdown_connections 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 1 -- 172.21.15.144:0/3971613223 wait complete. 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f68316ecb80 20 client.5697 trim_cache size 0 max 16384 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: "cluster_fsid": "af726c13-c3f1-4279-9fca-12c6e4621b18", 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: "whoami": 0, 2023-11-28T11:53:00.572 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 5535, 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "want_state": "up:active", 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "state": "up:active", 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs", 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "rank_uptime": 10.354175157, 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "mdsmap_epoch": 24, 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch": 46, 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch_barrier": 46, 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout: "uptime": 15.687235190000001 2023-11-28T11:53:00.573 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:00.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.562+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f680c0682f0 msgr2=0x7f680c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f680c0682f0 0x7f680c06a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f682006d920 tx=0x7f6814009210 comp rx=0 tx=0).stop 2023-11-28T11:53:00.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 msgr2=0x7f6820155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 0x7f6820155e40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6810009fd0 tx=0x7f6810004750 comp rx=0 tx=0).stop 2023-11-28T11:53:00.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 shutdown_connections 2023-11-28T11:53:00.574 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68200cdc80 0x7f6820155e40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f680c0682f0 0x7f680c06a7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68201e3c40 0x7f6820153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 --2- 172.21.15.144:0/2826810914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6820159490 0x7f6820154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 >> 172.21.15.144:0/2826810914 conn(0x7f68200cb670 msgr2=0x7f6820071560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 shutdown_connections 2023-11-28T11:53:00.575 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.563+0000 7f6825563700 1 -- 172.21.15.144:0/2826810914 wait complete. 2023-11-28T11:53:00.577 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:53:00.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.696+0000 7fe502b8e700 1 Processor -- start 2023-11-28T11:53:00.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.696+0000 7fe502b8e700 1 -- start start 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe502b8e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160b10 0x7fe4fc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe502b8e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc06bbf0 con 0x7fe4fc160740 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe502b8e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0c9e10 con 0x7fe4fc06b060 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe502b8e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc0ca720 con 0x7fe4fc160b10 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe50092a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe4fc160740 0x7fe4fc0cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54278/0 (socket says 172.21.15.144:54278) 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe50092a700 1 -- 172.21.15.144:0/4013964996 learned_addr learned my addr 172.21.15.144:0/4013964996 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fbfff700 1 --2- 172.21.15.144:0/4013964996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160b10 0x7fe4fc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320305604 0 0) 0x7fe4fc06bbf0 con 0x7fe4fc160740 2023-11-28T11:53:00.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ec0035b0 con 0x7fe4fc160740 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247136290 0 0) 0x7fe4fc0c9e10 con 0x7fe4fc06b060 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4fc06bbf0 con 0x7fe4fc06b060 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fbfff700 1 -- 172.21.15.144:0/4013964996 >> v1:172.21.15.177:6789/0 conn(0x7fe4fc06b060 legacy=0x7fe4fc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fbfff700 1 -- 172.21.15.144:0/4013964996 >> v1:172.21.15.144:6789/0 conn(0x7fe4fc160740 legacy=0x7fe4fc0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3089201782 0 0) 0x7fe4ec0035b0 con 0x7fe4fc160740 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.697+0000 7fe4fbfff700 1 -- 172.21.15.144:0/4013964996 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc06efc0 con 0x7fe4fc160b10 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe4fbfff700 1 --2- 172.21.15.144:0/4013964996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160b10 0x7fe4fc0cd390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009870 tx=0x7fe4e8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9eb745e7aa7e54 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0001670 con 0x7fe4fc160b10 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e8004030 con 0x7fe4fc160b10 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe4fb7fe700 1 -- 172.21.15.144:0/4013964996 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80043a0 con 0x7fe4fc160b10 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 -- 172.21.15.144:0/4013964996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160b10 msgr2=0x7fe4fc0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.702 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 --2- 172.21.15.144:0/4013964996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160b10 0x7fe4fc0cd390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009870 tx=0x7fe4e8009b80 comp rx=0 tx=0).stop 2023-11-28T11:53:00.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 -- 172.21.15.144:0/4013964996 shutdown_connections 2023-11-28T11:53:00.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 --2- 172.21.15.144:0/4013964996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160b10 0x7fe4fc0cd390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 -- 172.21.15.144:0/4013964996 >> 172.21.15.144:0/4013964996 conn(0x7fe4fc0b12b0 msgr2=0x7fe4fc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 -- 172.21.15.144:0/4013964996 shutdown_connections 2023-11-28T11:53:00.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.698+0000 7fe502b8e700 1 -- 172.21.15.144:0/4013964996 wait complete. 2023-11-28T11:53:00.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.700+0000 7fe502b8e700 1 Processor -- start 2023-11-28T11:53:00.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.700+0000 7fe502b8e700 1 -- start start 2023-11-28T11:53:00.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe502b8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 0x7fe4fc146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe502b8e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160740 0x7fe4fc06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe502b8e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4fc160b10 0x7fe4fc0ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe502b8e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f6850 con 0x7fe4fc06b060 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe502b8e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc06e320 con 0x7fe4fc160b10 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe502b8e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f5cd0 con 0x7fe4fc160740 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe50092a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 0x7fe4fc146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe50092a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 0x7fe4fc146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37572/0 (socket says 172.21.15.144:37572) 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe50092a700 1 -- 172.21.15.144:0/3850283584 learned_addr learned my addr 172.21.15.144:0/3850283584 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe50112b700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4fc160b10 0x7fe4fc0ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.701+0000 7fe4fbfff700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160740 0x7fe4fc06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe50092a700 1 -- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160740 msgr2=0x7fe4fc06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.706 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe50092a700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160740 0x7fe4fc06dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe50092a700 1 -- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4fc160b10 msgr2=0x7fe4fc0ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe50092a700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4fc160b10 0x7fe4fc0ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe50092a700 1 -- 172.21.15.144:0/3850283584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80094a0 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe50092a700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 0x7fe4fc146d40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c750 tx=0x7fe4f000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0004c80 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0004e00 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0007310 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc0ca3b0 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.702+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc0ca0d0 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.703+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc0cd8d0 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.703+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4f001a020 con 0x7fe4fc06b060 2023-11-28T11:53:00.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.704+0000 7fe4f97fa700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4e0068320 0x7fe4e006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:00.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.704+0000 7fe4fbfff700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4e0068320 0x7fe4e006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:00.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.704+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4f0085870 con 0x7fe4fc06b060 2023-11-28T11:53:00.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.704+0000 7fe4fbfff700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4e0068320 0x7fe4e006a7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80090b0 tx=0x7fe4e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:00.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.708+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4f00520d0 con 0x7fe4fc06b060 2023-11-28T11:53:00.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.934+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4fc0b5740 con 0x7fe4fc06b060 2023-11-28T11:53:00.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.934+0000 7fe4f97fa700 1 -- 172.21.15.144:0/3850283584 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe4f00556f0 con 0x7fe4fc06b060 2023-11-28T11:53:00.936 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:53:00.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.936+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4e0068320 msgr2=0x7fe4e006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.938 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.936+0000 7fe502b8e700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4e0068320 0x7fe4e006a7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80090b0 tx=0x7fe4e8008030 comp rx=0 tx=0).stop 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 msgr2=0x7fe4fc146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 0x7fe4fc146d40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c750 tx=0x7fe4f000ca60 comp rx=0 tx=0).stop 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 shutdown_connections 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4fc06b060 0x7fe4fc146d40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4e0068320 0x7fe4e006a7d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4fc160b10 0x7fe4fc0ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 --2- 172.21.15.144:0/3850283584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe4fc160740 0x7fe4fc06dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 >> 172.21.15.144:0/3850283584 conn(0x7fe4fc0b12b0 msgr2=0x7fe4fc06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 shutdown_connections 2023-11-28T11:53:00.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:00.937+0000 7fe502b8e700 1 -- 172.21.15.144:0/3850283584 wait complete. 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:53:00.951 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:53:00.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:00.952 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-11-28T11:53:00.969 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:53:00.969 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:53:00.969 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:53:00.969 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.969 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:53:00.969 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42321sec preferred_lft 42321sec 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1121:9a48:f2a:7a1a/64 scope link noprefixroute 2023-11-28T11:53:00.970 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1a:fa:92:22:2e:a9 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::fccf:aaff:fe54:493c/64 scope link 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether ee:83:c8:ce:35:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec83:c8ff:fece:355d/64 scope link 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:53:00.971 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1a:fa:92:22:2e:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:53:00.972 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::18fa:92ff:fe22:2ea9/64 scope link 2023-11-28T11:53:00.972 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:00.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:00.972 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:53:01.035 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T11:53:01.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.036 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-11-28T11:53:01.098 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-11-28T11:53:01.098 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:53:01.098 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:53:01.098 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:53:01.099 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:53:01.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.099 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:53:01.099 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:53:01.099 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:53:01.099 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:53:01.100 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:53:01.100 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:53:01.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.204 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:53:01.220 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:53:01.221 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:53:01.221 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:53:01.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.221 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-11-28T11:53:01.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:53:01.287 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:01.344 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:53:01.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:53:01.345 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:01.401 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:53:01.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.401 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:53:01.486 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:53:01.486 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:53:01.487 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:53:01.495 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:53:01.496 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:53:01.496 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:53:01.496 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:53:01.496 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:53:01.562 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:53:01.563 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:53:01.565 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:53:01.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:01.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.592 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:53:01.677 INFO:teuthology.orchestra.run.smithi144.stdout:client5066 2023-11-28T11:53:01.677 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:53:01.677 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:01.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.694 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T11:53:01.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:01.758 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T11:53:01.818 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:53:01.818 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:53:01.818 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:53:01.818 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:53:01.818 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T11:53:01.818 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:53:01.923 INFO:teuthology.orchestra.run.smithi144.stdout:1099511627777 2023-11-28T11:53:02.113 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.271+0000 7f317ceab700 1 Processor -- start 2023-11-28T11:53:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.271+0000 7f317ceab700 1 -- start start 2023-11-28T11:53:02.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f317ceab700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f317815cef0 0x7f31780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f317ceab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31780cbdf0 0x7f317806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f317ceab700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f317806b8c0 0x7f3178071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f317ceab700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317806bd40 con 0x7f317806b8c0 2023-11-28T11:53:02.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f317ceab700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31780cc5f0 con 0x7f317815cef0 2023-11-28T11:53:02.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f317ceab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317815d470 con 0x7f31780cbdf0 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.272+0000 7f3176d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f317806b8c0 0x7f3178071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f317806b8c0 0x7f3178071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37588/0 (socket says 172.21.15.144:37588) 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 -- 172.21.15.144:0/1317635765 learned_addr learned my addr 172.21.15.144:0/1317635765 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 -- 172.21.15.144:0/1317635765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31780cbdf0 msgr2=0x7f317806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3175d9b700 1 --2- 172.21.15.144:0/1317635765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31780cbdf0 0x7f317806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f317659c700 1 --2- 172.21.15.144:0/1317635765 >> v2:172.21.15.177:3300/0 conn(0x7f317815cef0 0x7f31780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 --2- 172.21.15.144:0/1317635765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31780cbdf0 0x7f317806b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 -- 172.21.15.144:0/1317635765 >> v2:172.21.15.177:3300/0 conn(0x7f317815cef0 msgr2=0x7f31780af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 --2- 172.21.15.144:0/1317635765 >> v2:172.21.15.177:3300/0 conn(0x7f317815cef0 0x7f31780af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 -- 172.21.15.144:0/1317635765 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317815d750 con 0x7f317806b8c0 2023-11-28T11:53:02.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f3176d9d700 1 --2- 172.21.15.144:0/1317635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f3178071040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f31780723f0 tx=0x7f317000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a4d95c70e2ed35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f317559a700 1 -- 172.21.15.144:0/1317635765 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3170010070 con 0x7f317806b8c0 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f317559a700 1 -- 172.21.15.144:0/1317635765 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317000dd90 con 0x7f317806b8c0 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.273+0000 7f317559a700 1 -- 172.21.15.144:0/1317635765 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317000b310 con 0x7f317806b8c0 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 -- 172.21.15.144:0/1317635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 msgr2=0x7f3178071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 --2- 172.21.15.144:0/1317635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f3178071040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f31780723f0 tx=0x7f317000d430 comp rx=0 tx=0).stop 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 -- 172.21.15.144:0/1317635765 shutdown_connections 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 --2- 172.21.15.144:0/1317635765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f3178071040 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 --2- 172.21.15.144:0/1317635765 >> v2:172.21.15.177:3300/0 conn(0x7f317815cef0 0x7f31780af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 --2- 172.21.15.144:0/1317635765 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f31780cbdf0 0x7f317806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 -- 172.21.15.144:0/1317635765 >> 172.21.15.144:0/1317635765 conn(0x7f31780d0000 msgr2=0x7f31780d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 -- 172.21.15.144:0/1317635765 shutdown_connections 2023-11-28T11:53:02.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.274+0000 7f317ceab700 1 -- 172.21.15.144:0/1317635765 wait complete. 2023-11-28T11:53:02.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.275+0000 7f317ceab700 1 Processor -- start 2023-11-28T11:53:02.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 -- start start 2023-11-28T11:53:02.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f31780c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31780cbdf0 0x7f31781598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317815cef0 0x7f31780b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31780c8ac0 con 0x7f317806b8c0 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31780c8da0 con 0x7f31780cbdf0 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317ceab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31780c9080 con 0x7f317815cef0 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f31780c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f31780c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37590/0 (socket says 172.21.15.144:37590) 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.276+0000 7f317659c700 1 -- 172.21.15.144:0/2939874493 learned_addr learned my addr 172.21.15.144:0/2939874493 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f3175d9b700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31780cbdf0 0x7f31781598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f3176d9d700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317815cef0 0x7f31780b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317659c700 1 -- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317815cef0 msgr2=0x7f31780b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317659c700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317815cef0 0x7f31780b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317659c700 1 -- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31780cbdf0 msgr2=0x7f31781598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317659c700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31780cbdf0 0x7f31781598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317659c700 1 -- 172.21.15.144:0/2939874493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317000bd40 con 0x7f317806b8c0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317659c700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f31780c9b60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c7f0 tx=0x7f316c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c011800 con 0x7f317806b8c0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c011960 con 0x7f317806b8c0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317815dd10 con 0x7f317806b8c0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c011cd0 con 0x7f317806b8c0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.277+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317815da30 con 0x7f317806b8c0 2023-11-28T11:53:02.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.278+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317815dff0 con 0x7f317806b8c0 2023-11-28T11:53:02.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.279+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f316c017030 con 0x7f317806b8c0 2023-11-28T11:53:02.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.279+0000 7f3166ffd700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31580682f0 0x7f315806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.279+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f316c0855b0 con 0x7f317806b8c0 2023-11-28T11:53:02.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.279+0000 7f3175d9b700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31580682f0 0x7f315806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.279+0000 7f3175d9b700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31580682f0 0x7f315806a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f317809d730 tx=0x7f3168008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.283+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f316c051e10 con 0x7f317806b8c0 2023-11-28T11:53:02.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.508+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31780d0e60 con 0x7f317806b8c0 2023-11-28T11:53:02.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.508+0000 7f3166ffd700 1 -- 172.21.15.144:0/2939874493 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f316c056e60 con 0x7f317806b8c0 2023-11-28T11:53:02.510 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:02.511 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:53:02.512 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.510+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31580682f0 msgr2=0x7f315806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.512 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.510+0000 7f317ceab700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31580682f0 0x7f315806a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f317809d730 tx=0x7f3168008040 comp rx=0 tx=0).stop 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.510+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 msgr2=0x7f31780c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.510+0000 7f317ceab700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f31780c9b60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c7f0 tx=0x7f316c00cb00 comp rx=0 tx=0).stop 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.510+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 shutdown_connections 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f317806b8c0 0x7f31780c9b60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f31580682f0 0x7f315806a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f317815cef0 0x7f31780b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 --2- 172.21.15.144:0/2939874493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31780cbdf0 0x7f31781598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 >> 172.21.15.144:0/2939874493 conn(0x7f31780d0000 msgr2=0x7f317806dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 shutdown_connections 2023-11-28T11:53:02.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.511+0000 7f317ceab700 1 -- 172.21.15.144:0/2939874493 wait complete. 2023-11-28T11:53:02.514 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:02.525 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:53:02.685 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2023-11-28T11:53:02.685 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:53:02.685 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:53:02.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.729+0000 7f7f7cc67080 1 Processor -- start 2023-11-28T11:53:02.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 -- start start 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b1d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 0x561e0b216420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b2ab300 0x561e0b217180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b110fe0 con 0x561e0b215100 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b220ef0 con 0x561e0b2ab300 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7cc67080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e0b2aec80 con 0x561e0b1d75f0 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f72569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 0x561e0b216420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f72569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 0x561e0b216420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37606/0 (socket says 172.21.15.144:37606) 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f72569700 1 -- 172.21.15.144:0/3488396157 learned_addr learned my addr 172.21.15.144:0/3488396157 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f72d6a700 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b1d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.730+0000 7f7f7356b700 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b2ab300 0x561e0b217180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f72569700 1 -- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 msgr2=0x561e0b1d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f72569700 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b1d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f72569700 1 -- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b2ab300 msgr2=0x561e0b217180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f72569700 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b2ab300 0x561e0b217180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f72569700 1 -- 172.21.15.144:0/3488396157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f68005de0 con 0x561e0b215100 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f72569700 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 0x561e0b216420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800a4f0 tx=0x7f7f6800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dade756b1a6d4a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f71567700 1 -- 172.21.15.144:0/3488396157 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68008690 con 0x561e0b215100 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f71567700 1 -- 172.21.15.144:0/3488396157 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68008ba0 con 0x561e0b215100 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f71567700 1 -- 172.21.15.144:0/3488396157 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6801e450 con 0x561e0b215100 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3488396157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 msgr2=0x561e0b216420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 0x561e0b216420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800a4f0 tx=0x7f7f6800d000 comp rx=0 tx=0).stop 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3488396157 shutdown_connections 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b2ab300 0x561e0b217180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b1d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3488396157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b215100 0x561e0b216420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3488396157 >> 172.21.15.144:0/3488396157 conn(0x561e0b202b60 msgr2=0x561e0b2064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3488396157 shutdown_connections 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.731+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3488396157 wait complete. 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.732+0000 7f7f7cc67080 1 Processor -- start 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.732+0000 7f7f7cc67080 1 -- start start 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b2a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b215100 0x561e0b2ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b2ab300 0x561e0b2ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b2adbb0 con 0x561e0b2ab300 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b2aa1f0 con 0x561e0b215100 2023-11-28T11:53:02.743 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e0b21fe80 con 0x561e0b1d75f0 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b2ab300 0x561e0b2ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b2ab300 0x561e0b2ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37614/0 (socket says 172.21.15.144:37614) 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 -- 172.21.15.144:0/3956184950 learned_addr learned my addr 172.21.15.144:0/3956184950 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f72569700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b215100 0x561e0b2ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f72d6a700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b2a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 -- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 msgr2=0x561e0b2a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b2a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 -- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b215100 msgr2=0x561e0b2ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e0b215100 0x561e0b2ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 -- 172.21.15.144:0/3956184950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0b1434f0 con 0x561e0b2ab300 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f72d6a700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x561e0b1d75f0 0x561e0b2a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7356b700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b2ab300 0x561e0b2ad670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0108e0 tx=0x7f7f6c010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f5bfff700 1 -- 172.21.15.144:0/3956184950 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c012670 con 0x561e0b2ab300 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f5bfff700 1 -- 172.21.15.144:0/3956184950 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c0127d0 con 0x561e0b2ab300 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f5bfff700 1 -- 172.21.15.144:0/3956184950 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c012b60 con 0x561e0b2ab300 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e0b129120 con 0x561e0b2ab300 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.733+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e0b11c830 con 0x561e0b2ab300 2023-11-28T11:53:02.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f5bfff700 1 -- 172.21.15.144:0/3956184950 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7f6c024070 con 0x561e0b2ab300 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f5bfff700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7f60066c10 0x7f7f600690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f5bfff700 1 -- 172.21.15.144:0/3956184950 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f6c084f30 con 0x561e0b2ab300 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f72d6a700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7f60066c10 0x7f7f600690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x561e0b1d5ba0 0x561e0b21f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) v8 -- 0x561e0b2a98e0 con 0x561e0b1d5ba0 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f72569700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x561e0b1d5ba0 0x561e0b21f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.735+0000 7f7f72d6a700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7f60066c10 0x7f7f600690c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000c00 tx=0x7f7f64001720 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f72569700 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x561e0b1d5ba0 0x561e0b21f480 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f72569700 1 -- 172.21.15.144:0/3956184950 <== osd.1 v2:172.21.15.144:6808/103557 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f7f68019040 con 0x561e0b1d5ba0 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x561e0b1d5ba0 msgr2=0x561e0b21f480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x561e0b1d5ba0 0x561e0b21f480 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7f60066c10 msgr2=0x7f7f600690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7f60066c10 0x7f7f600690c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000c00 tx=0x7f7f64001720 comp rx=0 tx=0).stop 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b2ab300 msgr2=0x561e0b2ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 --2- 172.21.15.144:0/3956184950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0b2ab300 0x561e0b2ad670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0108e0 tx=0x7f7f6c010bf0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7356b700 1 -- 172.21.15.144:0/3956184950 reap_dead start 2023-11-28T11:53:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 shutdown_connections 2023-11-28T11:53:02.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 >> 172.21.15.144:0/3956184950 conn(0x561e0b202b60 msgr2=0x561e0b206330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:02.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 shutdown_connections 2023-11-28T11:53:02.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.736+0000 7f7f7cc67080 1 -- 172.21.15.144:0/3956184950 wait complete. 2023-11-28T11:53:02.749 INFO:teuthology.orchestra.run.smithi144.stdout:Yalphaparent_a 2023-11-28T11:53:02.749 DEBUG:teuthology.orchestra.run.smithi144:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:53:02.857 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:02.857 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 1099511627777, 2023-11-28T11:53:02.857 INFO:teuthology.orchestra.run.smithi144.stdout: "ancestors": [ 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: "dirino": 1099511627776, 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: "dname": "alpha", 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: "version": 2 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: "dirino": 1, 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: "dname": "parent_a", 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: "version": 4 2023-11-28T11:53:02.858 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:02.859 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2023-11-28T11:53:02.859 INFO:teuthology.orchestra.run.smithi144.stdout: "pool": 8, 2023-11-28T11:53:02.859 INFO:teuthology.orchestra.run.smithi144.stdout: "old_pools": [] 2023-11-28T11:53:02.859 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:02.859 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:02.860 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T11:53:02.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.903+0000 7f6c5ccd8080 1 Processor -- start 2023-11-28T11:53:02.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.903+0000 7f6c5ccd8080 1 -- start start 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c5ccd8080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 0x5558fbb7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c5ccd8080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbc55460 0x5558fbc52ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c5ccd8080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c5ccd8080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fbb25000 con 0x5558fbbc79e0 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c5ccd8080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fbab7fe0 con 0x5558fbc55460 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c5ccd8080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558fbbc36d0 con 0x5558fbc59df0 2023-11-28T11:53:02.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 0x5558fbb7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 0x5558fbb7cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37626/0 (socket says 172.21.15.144:37626) 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 -- 172.21.15.144:0/2535180273 learned_addr learned my addr 172.21.15.144:0/2535180273 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c525da700 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbc55460 0x5558fbc52ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c535dc700 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 -- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 msgr2=0x5558fbc531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 -- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbc55460 msgr2=0x5558fbc52ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbc55460 0x5558fbc52ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 -- 172.21.15.144:0/2535180273 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44005d20 con 0x5558fbbc79e0 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.904+0000 7f6c52ddb700 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 0x5558fbb7cc90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6c48001eb0 tx=0x7f6c4800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d93eb293e1aa3ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c515d8700 1 -- 172.21.15.144:0/2535180273 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800f3a0 con 0x5558fbbc79e0 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c515d8700 1 -- 172.21.15.144:0/2535180273 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4800a040 con 0x5558fbbc79e0 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c515d8700 1 -- 172.21.15.144:0/2535180273 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800f740 con 0x5558fbbc79e0 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/2535180273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 msgr2=0x5558fbb7cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 0x5558fbb7cc90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6c48001eb0 tx=0x7f6c4800b4a0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/2535180273 shutdown_connections 2023-11-28T11:53:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbbc79e0 0x5558fbb7cc90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/2535180273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbc55460 0x5558fbc52ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/2535180273 >> 172.21.15.144:0/2535180273 conn(0x5558fbba9b70 msgr2=0x5558fbc54750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/2535180273 shutdown_connections 2023-11-28T11:53:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.905+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/2535180273 wait complete. 2023-11-28T11:53:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 Processor -- start 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 -- start start 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbbc79e0 0x5558fbc571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbc55460 0x5558fbc46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc57750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fbc54750 con 0x5558fbc55460 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fbc47130 con 0x5558fbbc79e0 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c5ccd8080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558fbc549f0 con 0x5558fbc59df0 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c525da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbc55460 0x5558fbc46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c525da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbc55460 0x5558fbc46bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37634/0 (socket says 172.21.15.144:37634) 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c525da700 1 -- 172.21.15.144:0/3106558609 learned_addr learned my addr 172.21.15.144:0/3106558609 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:02.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.906+0000 7f6c52ddb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbbc79e0 0x5558fbc571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c535dc700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc57750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c525da700 1 -- 172.21.15.144:0/3106558609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 msgr2=0x5558fbc57750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c525da700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5558fbc59df0 0x5558fbc57750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c525da700 1 -- 172.21.15.144:0/3106558609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbbc79e0 msgr2=0x5558fbc571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c525da700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5558fbbc79e0 0x5558fbc571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c525da700 1 -- 172.21.15.144:0/3106558609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48001ae0 con 0x5558fbc55460 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c525da700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbc55460 0x5558fbc46bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6c44002000 tx=0x7f6c44003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c3bfff700 1 -- 172.21.15.144:0/3106558609 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44008690 con 0x5558fbc55460 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c3bfff700 1 -- 172.21.15.144:0/3106558609 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c44013690 con 0x5558fbc55460 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558fbac3830 con 0x5558fbc55460 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c3bfff700 1 -- 172.21.15.144:0/3106558609 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44013a20 con 0x5558fbc55460 2023-11-28T11:53:02.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.907+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558fbad0360 con 0x5558fbc55460 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.908+0000 7f6c3bfff700 1 -- 172.21.15.144:0/3106558609 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c44013b80 con 0x5558fbc55460 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c3bfff700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c40066c10 0x7f6c400690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c3bfff700 1 -- 172.21.15.144:0/3106558609 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c44085fb0 con 0x5558fbc55460 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c52ddb700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c40066c10 0x7f6c400690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x5558fbbbe690 0x5558fbcf7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v8 -- 0x5558fbcf8140 con 0x5558fbbbe690 2023-11-28T11:53:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c535dc700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x5558fbbbe690 0x5558fbcf7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c52ddb700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c40066c10 0x7f6c400690c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b4e0 tx=0x7f6c48003000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.909+0000 7f6c535dc700 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x5558fbbbe690 0x5558fbcf7c00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c535dc700 1 -- 172.21.15.144:0/3106558609 <== osd.1 v2:172.21.15.144:6808/103557 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v46'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6c4c001630 con 0x5558fbbbe690 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x5558fbbbe690 msgr2=0x5558fbcf7c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x5558fbbbe690 0x5558fbcf7c00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c40066c10 msgr2=0x7f6c400690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6c40066c10 0x7f6c400690c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b4e0 tx=0x7f6c48003000 comp rx=0 tx=0).stop 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbc55460 msgr2=0x5558fbc46bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 --2- 172.21.15.144:0/3106558609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5558fbc55460 0x5558fbc46bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6c44002000 tx=0x7f6c44003020 comp rx=0 tx=0).stop 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c535dc700 1 -- 172.21.15.144:0/3106558609 reap_dead start 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 shutdown_connections 2023-11-28T11:53:02.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 >> 172.21.15.144:0/3106558609 conn(0x5558fbba9b70 msgr2=0x5558fbc596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:02.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 shutdown_connections 2023-11-28T11:53:02.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:02.911+0000 7f6c5ccd8080 1 -- 172.21.15.144:0/3106558609 wait complete. 2023-11-28T11:53:02.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T11:53:02.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T11:53:03.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.045+0000 7fea21cba700 1 Processor -- start 2023-11-28T11:53:03.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.045+0000 7fea21cba700 1 -- start start 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea21cba700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea1c0af060 0x7fea1c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea21cba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c06a980 0x7fea1c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea21cba700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c0713c0 con 0x7fea1c0af430 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea21cba700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c06dee0 con 0x7fea1c0af060 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea21cba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0716a0 con 0x7fea1c06a980 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea1bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c06a980 0x7fea1c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea1b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea1c0af060 0x7fea1c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea1affd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fea1c0af430 0x7fea1c06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:54844/0 (socket says 172.21.15.144:54844) 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.046+0000 7fea1affd700 1 -- 172.21.15.144:0/3969563947 learned_addr learned my addr 172.21.15.144:0/3969563947 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:03.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1a7fc700 1 -- 172.21.15.144:0/3969563947 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048737775 0 0) 0x7fea1c0713c0 con 0x7fea1c0af430 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1a7fc700 1 -- 172.21.15.144:0/3969563947 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea080035b0 con 0x7fea1c0af430 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1bfff700 1 -- 172.21.15.144:0/3969563947 >> v2:172.21.15.177:3300/0 conn(0x7fea1c0af060 msgr2=0x7fea1c070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1bfff700 1 --2- 172.21.15.144:0/3969563947 >> v2:172.21.15.177:3300/0 conn(0x7fea1c0af060 0x7fea1c070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1bfff700 1 -- 172.21.15.144:0/3969563947 >> v1:172.21.15.177:6789/0 conn(0x7fea1c0af430 legacy=0x7fea1c06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1bfff700 1 -- 172.21.15.144:0/3969563947 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c071980 con 0x7fea1c06a980 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/3969563947 >> v2:172.21.15.177:3300/0 conn(0x7fea1c0af060 0x7fea1c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1bfff700 1 --2- 172.21.15.144:0/3969563947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c06a980 0x7fea1c06d9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea14009840 tx=0x7fea14009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fdcc5fd8f2452e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1a7fc700 1 -- 172.21.15.144:0/3969563947 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14011830 con 0x7fea1c06a980 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1a7fc700 1 -- 172.21.15.144:0/3969563947 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14011990 con 0x7fea1c06a980 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.047+0000 7fea1a7fc700 1 -- 172.21.15.144:0/3969563947 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14011d20 con 0x7fea1c06a980 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 -- 172.21.15.144:0/3969563947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c06a980 msgr2=0x7fea1c06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 --2- 172.21.15.144:0/3969563947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c06a980 0x7fea1c06d9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea14009840 tx=0x7fea14009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:03.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 -- 172.21.15.144:0/3969563947 shutdown_connections 2023-11-28T11:53:03.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 --2- 172.21.15.144:0/3969563947 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c06a980 0x7fea1c06d9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 --2- 172.21.15.144:0/3969563947 >> v2:172.21.15.177:3300/0 conn(0x7fea1c0af060 0x7fea1c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 -- 172.21.15.144:0/3969563947 >> 172.21.15.144:0/3969563947 conn(0x7fea1c06b380 msgr2=0x7fea1c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 -- 172.21.15.144:0/3969563947 shutdown_connections 2023-11-28T11:53:03.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.048+0000 7fea21cba700 1 -- 172.21.15.144:0/3969563947 wait complete. 2023-11-28T11:53:03.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.049+0000 7fea21cba700 1 Processor -- start 2023-11-28T11:53:03.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.049+0000 7fea21cba700 1 -- start start 2023-11-28T11:53:03.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea21cba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 0x7fea1c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea21cba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c0af060 0x7fea1c15f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea21cba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea1c0af430 0x7fea1c0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea21cba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15e420 con 0x7fea1c06a980 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea21cba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15e700 con 0x7fea1c0af430 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea21cba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c15e9e0 con 0x7fea1c0af060 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 0x7fea1c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 0x7fea1c14fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37646/0 (socket says 172.21.15.144:37646) 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 -- 172.21.15.144:0/4037069192 learned_addr learned my addr 172.21.15.144:0/4037069192 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1affd700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c0af060 0x7fea1c15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1bfff700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea1c0af430 0x7fea1c0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 -- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c0af060 msgr2=0x7fea1c15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c0af060 0x7fea1c15f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 -- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea1c0af430 msgr2=0x7fea1c0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea1c0af430 0x7fea1c0b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.050+0000 7fea1b7fe700 1 -- 172.21.15.144:0/4037069192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c009a70 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.051+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 0x7fea1c14fad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00efc0 tx=0x7fea0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.051+0000 7fea13fff700 1 -- 172.21.15.144:0/4037069192 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00cdd0 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.051+0000 7fea13fff700 1 -- 172.21.15.144:0/4037069192 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c00f040 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.051+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea140094a0 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.051+0000 7fea13fff700 1 -- 172.21.15.144:0/4037069192 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0146a0 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.051+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c071f40 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.052+0000 7fea13fff700 1 -- 172.21.15.144:0/4037069192 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea0c014800 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.052+0000 7fea13fff700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9fc0682f0 0x7fe9fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.053+0000 7fea13fff700 1 -- 172.21.15.144:0/4037069192 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea0c086b80 con 0x7fea1c06a980 2023-11-28T11:53:03.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.053+0000 7fea1affd700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9fc0682f0 0x7fe9fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.053+0000 7fea1affd700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9fc0682f0 0x7fe9fc06a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b57d0 tx=0x7fea04007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.055+0000 7fea2de43b80 1 Processor -- start 2023-11-28T11:53:03.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.055+0000 7fea2de43b80 1 -- start start 2023-11-28T11:53:03.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c875baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c8785cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c87a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea2de43b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8645c90 con 0x55b7c87858e0 2023-11-28T11:53:03.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c86b4420 con 0x55b7c8783200 2023-11-28T11:53:03.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c86ad950 con 0x55b7c87861f0 2023-11-28T11:53:03.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c875baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c875baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37654/0 (socket says 172.21.15.144:37654) 2023-11-28T11:53:03.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 -- 172.21.15.144:0/860596112 learned_addr learned my addr 172.21.15.144:0/860596112 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1bfff700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c87a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1affd700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c8785cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 -- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 msgr2=0x55b7c87a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c87a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 -- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 msgr2=0x55b7c8785cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c8785cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 -- 172.21.15.144:0/860596112 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7c852f6b0 con 0x55b7c87858e0 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1affd700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c8785cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1bfff700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c87a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c875baf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea0c056a40 tx=0x7fea0c05f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1316a9a09d5a7f20 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.056+0000 7fea117fa700 1 -- 172.21.15.144:0/860596112 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c06e300 con 0x55b7c87858e0 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea117fa700 1 -- 172.21.15.144:0/860596112 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c0566a0 con 0x55b7c87858e0 2023-11-28T11:53:03.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea117fa700 1 -- 172.21.15.144:0/860596112 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c05edb0 con 0x55b7c87858e0 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 -- 172.21.15.144:0/860596112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 msgr2=0x55b7c875baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c875baf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea0c056a40 tx=0x7fea0c05f040 comp rx=0 tx=0).stop 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 -- 172.21.15.144:0/860596112 shutdown_connections 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c875baf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c87a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 --2- 172.21.15.144:0/860596112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c8785cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 -- 172.21.15.144:0/860596112 >> 172.21.15.144:0/860596112 conn(0x55b7c8758e30 msgr2=0x55b7c87597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 -- 172.21.15.144:0/860596112 shutdown_connections 2023-11-28T11:53:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.057+0000 7fea2de43b80 1 -- 172.21.15.144:0/860596112 wait complete. 2023-11-28T11:53:03.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.058+0000 7fea2de43b80 1 -- start start 2023-11-28T11:53:03.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.058+0000 7fea2de43b80 1 Processor -- start 2023-11-28T11:53:03.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c8783200 0x55b7c87a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c8784ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c8799910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c85319a0 con 0x55b7c87858e0 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8780350 con 0x55b7c87861f0 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c87a6b00 con 0x55b7c8783200 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea2de43b80 10 client.0 _ll_register_callbacks cb 0x55b7c8553310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c8784ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c8784ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37662/0 (socket says 172.21.15.144:37662) 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 -- 172.21.15.144:0/1427397408 learned_addr learned my addr 172.21.15.144:0/1427397408 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c8783200 0x55b7c87a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1bfff700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c8799910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 -- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c8783200 msgr2=0x55b7c87a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c8783200 0x55b7c87a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 -- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 msgr2=0x55b7c8799910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c8799910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 -- 172.21.15.144:0/1427397408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c071f40 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c8783200 0x55b7c87a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fea1affd700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c8784ac0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fea04006a80 tx=0x7fea04001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35785a018ad98314 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.059+0000 7fe9fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/1427397408 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04026070 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/1427397408 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea0401b070 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7c867b210 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/1427397408 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea04001580 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 10 client.5742 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b7c84dbe60 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/1427397408 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04030020 con 0x55b7c87858e0 2023-11-28T11:53:03.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/1427397408 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fea04029020 con 0x55b7c87858e0 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 10 client.5742 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 10 client.5742 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 20 client.5742 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c877f990 0x55b7c87b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 4 client.5742 mds_command: new command op to 5535 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7c8531fe0 con 0x55b7c877f990 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.060+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c877f990 0x55b7c87b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fe9fa7fc700 10 client.5742 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c877f990 0x55b7c87b35a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6d58be2c058161 server_cookie=d2e1818eda7f0923 in_seq=0 out_seq=0 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/1427397408 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55b7c8531fe0 con 0x55b7c877f990 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fe9fa7fc700 10 client.5742 handle_command_reply: tid=0 2023-11-28T11:53:03.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 client.5742 shutdown 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 msgr2=0x55b7c8784ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c8784ac0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fea04006a80 tx=0x7fea04001150 comp rx=0 tx=0).stop 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 shutdown_connections 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c877f990 0x55b7c87b35a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c8799910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c8783200 0x55b7c87a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 --2- 172.21.15.144:0/1427397408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c8784ac0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fe9fa7fc700 0 client.5742 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.061+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 >> 172.21.15.144:0/1427397408 conn(0x55b7c8758e30 msgr2=0x55b7c8756010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.062+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 shutdown_connections 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.062+0000 7fea2de43b80 1 -- 172.21.15.144:0/1427397408 wait complete. 2023-11-28T11:53:03.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.062+0000 7fea2de43b80 20 client.5742 trim_cache size 0 max 16384 2023-11-28T11:53:03.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.069+0000 7f140fb66700 1 Processor -- start 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.069+0000 7f140fb66700 1 -- start start 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140fb66700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f14080d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140fb66700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14080af5b0 con 0x7f14080cd370 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140fb66700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14080cfa60 con 0x7f140806a270 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140fb66700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14080cf4a0 con 0x7f140815cbc0 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140d101700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f14080cd370 0x7f14080cebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54382/0 (socket says 172.21.15.144:54382) 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140d101700 1 -- 172.21.15.144:0/2262750687 learned_addr learned my addr 172.21.15.144:0/2262750687 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.070+0000 7f140d902700 1 --2- 172.21.15.144:0/2262750687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f14080d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140c900700 1 -- 172.21.15.144:0/2262750687 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266824230 0 0) 0x7f13f8001110 con 0x7f14080cd370 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140c900700 1 -- 172.21.15.144:0/2262750687 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f40035b0 con 0x7f14080cd370 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140c900700 1 -- 172.21.15.144:0/2262750687 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523665288 0 0) 0x7f14080cfa60 con 0x7f140806a270 2023-11-28T11:53:03.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140d902700 1 -- 172.21.15.144:0/2262750687 >> v1:172.21.15.177:6789/0 conn(0x7f140806a270 legacy=0x7f14081605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140d902700 1 -- 172.21.15.144:0/2262750687 >> v1:172.21.15.144:6789/0 conn(0x7f14080cd370 legacy=0x7f14080cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140d902700 1 -- 172.21.15.144:0/2262750687 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14080cf780 con 0x7f140815cbc0 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140d902700 1 --2- 172.21.15.144:0/2262750687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f14080d0570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1404009870 tx=0x7f1404009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c60870298fdeea1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140c900700 1 -- 172.21.15.144:0/2262750687 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404011830 con 0x7f140815cbc0 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140c900700 1 -- 172.21.15.144:0/2262750687 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404011990 con 0x7f140815cbc0 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140c900700 1 -- 172.21.15.144:0/2262750687 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404011d20 con 0x7f140815cbc0 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140fb66700 1 -- 172.21.15.144:0/2262750687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 msgr2=0x7f14080d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140fb66700 1 --2- 172.21.15.144:0/2262750687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f14080d0570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1404009870 tx=0x7f1404009b80 comp rx=0 tx=0).stop 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140fb66700 1 -- 172.21.15.144:0/2262750687 shutdown_connections 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140fb66700 1 --2- 172.21.15.144:0/2262750687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f14080d0570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.071+0000 7f140fb66700 1 -- 172.21.15.144:0/2262750687 >> 172.21.15.144:0/2262750687 conn(0x7f140806b380 msgr2=0x7f140806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.072+0000 7f140fb66700 1 -- 172.21.15.144:0/2262750687 shutdown_connections 2023-11-28T11:53:03.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.072+0000 7f140fb66700 1 -- 172.21.15.144:0/2262750687 wait complete. 2023-11-28T11:53:03.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 Processor -- start 2023-11-28T11:53:03.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 -- start start 2023-11-28T11:53:03.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f140806a270 0x7f14081f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14080cd370 0x7f140815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f140806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14081f2fb0 con 0x7f14080cd370 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c7340 con 0x7f140806a270 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.073+0000 7f140fb66700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c7620 con 0x7f140815cbc0 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14080cd370 0x7f140815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14080cd370 0x7f140815a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37676/0 (socket says 172.21.15.144:37676) 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 -- 172.21.15.144:0/733611692 learned_addr learned my addr 172.21.15.144:0/733611692 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d902700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f140806a270 0x7f14081f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140e103700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f140806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 -- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 msgr2=0x7f140806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f140815cbc0 0x7f140806b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 -- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f140806a270 msgr2=0x7f14081f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f140806a270 0x7f14081f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 -- 172.21.15.144:0/733611692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040094a0 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d902700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f140806a270 0x7f14081f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140d101700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14080cd370 0x7f140815a4a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c4d0 tx=0x7f13f800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8005d70 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80070d0 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8007480 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140fb66700 1 -- 172.21.15.144:0/733611692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14080d0050 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.074+0000 7f140fb66700 1 -- 172.21.15.144:0/733611692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14080cfd70 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.075+0000 7f140fb66700 1 -- 172.21.15.144:0/733611692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f14080d0ab0 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.076+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13f8019030 con 0x7f14080cd370 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.076+0000 7f13fe7fc700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13ec0682f0 0x7f13ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.080 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.076+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13f8085650 con 0x7f14080cd370 2023-11-28T11:53:03.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.076+0000 7f140d902700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13ec0682f0 0x7f13ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.076+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+506 (secure 0 0 0) 0x7f13f8085ad0 con 0x7f14080cd370 2023-11-28T11:53:03.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.076+0000 7f140fb66700 1 -- 172.21.15.144:0/733611692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f14080af5b0 con 0x7f14080cd370 2023-11-28T11:53:03.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.077+0000 7f140d902700 1 --2- 172.21.15.144:0/733611692 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13ec0682f0 0x7f13ec06a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f14040090b0 tx=0x7f140401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.079+0000 7fea2de43b80 1 Processor -- start 2023-11-28T11:53:03.081 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.079+0000 7fea2de43b80 1 -- start start 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 0x55b7c8759200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87858e0 0x55b7c87b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c87826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea2de43b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c8756cd0 con 0x55b7c8783200 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c875cc60 con 0x55b7c87861f0 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c87848d0 con 0x55b7c87858e0 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 0x55b7c8759200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 0x55b7c8759200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37682/0 (socket says 172.21.15.144:37682) 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 -- 172.21.15.144:0/2168291020 learned_addr learned my addr 172.21.15.144:0/2168291020 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1affd700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87858e0 0x55b7c87b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1bfff700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c87826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 -- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87858e0 msgr2=0x55b7c87b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87858e0 0x55b7c87b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 -- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 msgr2=0x55b7c87826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c87826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1b7fe700 1 -- 172.21.15.144:0/2168291020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7c8674210 con 0x55b7c8783200 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1affd700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87858e0 0x55b7c87b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.080+0000 7fea1bfff700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c87826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.081+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 0x55b7c8759200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00ef90 tx=0x7fea0c05c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b36c53ee4ead47 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.081+0000 7fea11ffb700 1 -- 172.21.15.144:0/2168291020 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c055410 con 0x55b7c8783200 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.081+0000 7fea11ffb700 1 -- 172.21.15.144:0/2168291020 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c0564a0 con 0x55b7c8783200 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.081+0000 7fea11ffb700 1 -- 172.21.15.144:0/2168291020 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c024bd0 con 0x55b7c8783200 2023-11-28T11:53:03.085 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.081+0000 7fea2de43b80 1 -- 172.21.15.144:0/2168291020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 msgr2=0x55b7c8759200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.081+0000 7fea2de43b80 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 0x55b7c8759200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00ef90 tx=0x7fea0c05c000 comp rx=0 tx=0).stop 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 -- 172.21.15.144:0/2168291020 shutdown_connections 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c8783200 0x55b7c8759200 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c87861f0 0x55b7c87826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 --2- 172.21.15.144:0/2168291020 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87858e0 0x55b7c87b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 -- 172.21.15.144:0/2168291020 >> 172.21.15.144:0/2168291020 conn(0x55b7c8758e30 msgr2=0x55b7c8756290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 -- 172.21.15.144:0/2168291020 shutdown_connections 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 -- 172.21.15.144:0/2168291020 wait complete. 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 -- start start 2023-11-28T11:53:03.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.082+0000 7fea2de43b80 1 Processor -- start 2023-11-28T11:53:03.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c87a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c87a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c8754520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c87a3790 con 0x55b7c87858e0 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7c87ebb60 con 0x55b7c8783200 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7c875bf60 con 0x55b7c87861f0 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea2de43b80 10 client.0 _ll_register_callbacks cb 0x55b7c8553310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c87a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c87a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37690/0 (socket says 172.21.15.144:37690) 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 -- 172.21.15.144:0/576570565 learned_addr learned my addr 172.21.15.144:0/576570565 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c87a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 -- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 msgr2=0x55b7c8754520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.090 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c8754520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 -- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 msgr2=0x55b7c87a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c87a2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 -- 172.21.15.144:0/576570565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c083040 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fea1affd700 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c87a3210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fea040011e0 tx=0x7fea04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f070a3a6245108c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.083+0000 7fe9fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/576570565 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04026040 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/576570565 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea04031080 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7c879a360 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/576570565 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea04001580 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 10 client.5754 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b7c85022d0 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/576570565 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b7c879a360 con 0x55b7c87858e0 2023-11-28T11:53:03.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/576570565 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fea04033070 con 0x55b7c87858e0 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 10 client.5754 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 10 client.5754 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 20 client.5754 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c8754a60 0x55b7c8806170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 4 client.5754 mds_command: new command op to 5535 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55b7c866b230 con 0x55b7c8754a60 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c8754a60 0x55b7c8806170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fe9fa7fc700 10 client.5754 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.084+0000 7fea1b7fe700 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c8754a60 0x55b7c8806170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe85c4ab0652734a server_cookie=45a02cfd1754498 in_seq=0 out_seq=0 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fe9fa7fc700 1 -- 172.21.15.144:0/576570565 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b7c866b230 con 0x55b7c8754a60 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fe9fa7fc700 10 client.5754 handle_command_reply: tid=0 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 client.5754 shutdown 2023-11-28T11:53:03.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 msgr2=0x55b7c87a3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c87a3210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fea040011e0 tx=0x7fea04014000 comp rx=0 tx=0).stop 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 shutdown_connections 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55b7c8754a60 0x55b7c8806170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b7c87861f0 0x55b7c8754520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b7c8783200 0x55b7c87a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fe9fa7fc700 0 client.5754 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 --2- 172.21.15.144:0/576570565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c87858e0 0x55b7c87a3210 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 >> 172.21.15.144:0/576570565 conn(0x55b7c8758e30 msgr2=0x55b7c875cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 shutdown_connections 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.085+0000 7fea2de43b80 1 -- 172.21.15.144:0/576570565 wait complete. 2023-11-28T11:53:03.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea2de43b80 20 client.5754 trim_cache size 0 max 16384 2023-11-28T11:53:03.094 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:03.094 INFO:teuthology.orchestra.run.smithi144.stdout: "return_code": 0, 2023-11-28T11:53:03.094 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_tag": "31574e77-d1b7-4f7e-9258-16fe2f4e3908", 2023-11-28T11:53:03.094 INFO:teuthology.orchestra.run.smithi144.stdout: "mode": "asynchronous" 2023-11-28T11:53:03.094 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9fc0682f0 msgr2=0x7fe9fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9fc0682f0 0x7fe9fc06a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b57d0 tx=0x7fea04007dc0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 msgr2=0x7fea1c14fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 0x7fea1c14fad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00efc0 tx=0x7fea0c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 shutdown_connections 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea1c06a980 0x7fea1c14fad0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe9fc0682f0 0x7fe9fc06a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea1c0af430 0x7fea1c0b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 --2- 172.21.15.144:0/4037069192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea1c0af060 0x7fea1c15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.096 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 >> 172.21.15.144:0/4037069192 conn(0x7fea1c06b380 msgr2=0x7fea1c0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.096 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 shutdown_connections 2023-11-28T11:53:03.096 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.086+0000 7fea21cba700 1 -- 172.21.15.144:0/4037069192 wait complete. 2023-11-28T11:53:03.096 INFO:tasks.ceph.mds.d.smithi177.stderr:2023-11-28T11:53:03.087+0000 7f3182904700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5066=0-4194304@1} caps={5066=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5066 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55d371a67700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:53:03.100 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:53:03.223 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.222+0000 7fc9850ae700 1 Processor -- start 2023-11-28T11:53:03.224 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.222+0000 7fc9850ae700 1 -- start start 2023-11-28T11:53:03.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc9850ae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 0x7fc980071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc9850ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98015e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc9850ae700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980071980 con 0x7fc980154e70 2023-11-28T11:53:03.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc9850ae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9800ce050 con 0x7fc9800cdbd0 2023-11-28T11:53:03.226 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc9850ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc980155240 con 0x7fc980071580 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97e59c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 0x7fc980071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:49782/0 (socket says 172.21.15.144:49782) 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97e59c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 0x7fc980071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:39972/0 (socket says 172.21.15.144:39972) 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 -- 172.21.15.144:0/1245822205 learned_addr learned my addr 172.21.15.144:0/1245822205 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97ed9d700 1 --1- 172.21.15.144:0/1245822205 >> v1:172.21.15.177:6789/0 conn(0x7fc980154e70 0x7fc9800cf360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:54898/0 (socket says 172.21.15.144:54898) 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 -- 172.21.15.144:0/1245822205 >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 msgr2=0x7fc980071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 --2- 172.21.15.144:0/1245822205 >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 0x7fc980071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 -- 172.21.15.144:0/1245822205 >> v1:172.21.15.177:6789/0 conn(0x7fc980154e70 legacy=0x7fc9800cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 -- 172.21.15.144:0/1245822205 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98015b100 con 0x7fc980071580 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97e59c700 1 --2- 172.21.15.144:0/1245822205 >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 0x7fc980071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.227 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.223+0000 7fc97f59e700 1 --2- 172.21.15.144:0/1245822205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98015e680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc978009e50 tx=0x7fc97800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87923c1505324d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc97dd9b700 1 -- 172.21.15.144:0/1245822205 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800e070 con 0x7fc980071580 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc97dd9b700 1 -- 172.21.15.144:0/1245822205 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97800b910 con 0x7fc980071580 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc97dd9b700 1 -- 172.21.15.144:0/1245822205 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800bcc0 con 0x7fc980071580 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 -- 172.21.15.144:0/1245822205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 msgr2=0x7fc98015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 --2- 172.21.15.144:0/1245822205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98015e680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc978009e50 tx=0x7fc97800b330 comp rx=0 tx=0).stop 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 -- 172.21.15.144:0/1245822205 shutdown_connections 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 --2- 172.21.15.144:0/1245822205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98015e680 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 --2- 172.21.15.144:0/1245822205 >> v2:172.21.15.177:3300/0 conn(0x7fc9800cdbd0 0x7fc980071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 -- 172.21.15.144:0/1245822205 >> 172.21.15.144:0/1245822205 conn(0x7fc9800d0000 msgr2=0x7fc9800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 -- 172.21.15.144:0/1245822205 shutdown_connections 2023-11-28T11:53:03.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.224+0000 7fc9850ae700 1 -- 172.21.15.144:0/1245822205 wait complete. 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 Processor -- start 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 -- start start 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9800cdbd0 0x7fc9800cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 0x7fc98014a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98006df20 con 0x7fc980154e70 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98014a9f0 con 0x7fc9800cdbd0 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc9850ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98006d3a0 con 0x7fc980071580 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.226+0000 7fc97f59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 0x7fc98014a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 0x7fc98014a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37706/0 (socket says 172.21.15.144:37706) 2023-11-28T11:53:03.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 -- 172.21.15.144:0/681857162 learned_addr learned my addr 172.21.15.144:0/681857162 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97e59c700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9800cdbd0 0x7fc9800cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 -- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 msgr2=0x7fc98006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98006cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 -- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9800cdbd0 msgr2=0x7fc9800cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9800cdbd0 0x7fc9800cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 -- 172.21.15.144:0/681857162 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740094a0 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97e59c700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9800cdbd0 0x7fc9800cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98006cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc97f59e700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 0x7fc98014a4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc980155380 tx=0x7fc978004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc9677fe700 1 -- 172.21.15.144:0/681857162 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978004800 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc9677fe700 1 -- 172.21.15.144:0/681857162 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978004960 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc9677fe700 1 -- 172.21.15.144:0/681857162 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97801c680 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc978009b30 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.227+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98015b6c0 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.229+0000 7fc9677fe700 1 -- 172.21.15.144:0/681857162 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc97802c070 con 0x7fc980154e70 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.229+0000 7fc9677fe700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc96c0682f0 0x7fc96c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.229+0000 7fc9677fe700 1 -- 172.21.15.144:0/681857162 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9780876e0 con 0x7fc980154e70 2023-11-28T11:53:03.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.229+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc96c0682f0 0x7fc96c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.234 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.229+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc96c0682f0 0x7fc96c06a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc970000f60 tx=0x7fc970007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.231+0000 7fc991237b80 1 Processor -- start 2023-11-28T11:53:03.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.231+0000 7fc991237b80 1 -- start start 2023-11-28T11:53:03.240 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x55567927aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc991237b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 0x5556792a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a5170 0x5556792c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc991237b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555679164c10 con 0x5556792a2180 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556791d33a0 con 0x5556792a4860 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556791cc8d0 con 0x5556792a5170 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 0x5556792a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 0x5556792a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37712/0 (socket says 172.21.15.144:37712) 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 -- 172.21.15.144:0/2041887305 learned_addr learned my addr 172.21.15.144:0/2041887305 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x55567927aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97f59e700 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a5170 0x5556792c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 -- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a5170 msgr2=0x5556792c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.241 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a5170 0x5556792c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 -- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 msgr2=0x55567927aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x55567927aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 -- 172.21.15.144:0/2041887305 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55567904e630 con 0x5556792a2180 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc97e59c700 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 0x5556792a4c30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc9740114e0 tx=0x7fc9740117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685f754c849fc0eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.232+0000 7fc964ff9700 1 -- 172.21.15.144:0/2041887305 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974004240 con 0x5556792a2180 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc964ff9700 1 -- 172.21.15.144:0/2041887305 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9740043a0 con 0x5556792a2180 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc964ff9700 1 -- 172.21.15.144:0/2041887305 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9740036a0 con 0x5556792a2180 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 -- 172.21.15.144:0/2041887305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 msgr2=0x5556792a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 0x5556792a4c30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc9740114e0 tx=0x7fc9740117f0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 -- 172.21.15.144:0/2041887305 shutdown_connections 2023-11-28T11:53:03.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a5170 0x5556792c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x55567927aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 --2- 172.21.15.144:0/2041887305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a2180 0x5556792a4c30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 -- 172.21.15.144:0/2041887305 >> 172.21.15.144:0/2041887305 conn(0x555679277db0 msgr2=0x555679278740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 -- 172.21.15.144:0/2041887305 shutdown_connections 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.233+0000 7fc991237b80 1 -- 172.21.15.144:0/2041887305 wait complete. 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.234+0000 7fc991237b80 1 -- start start 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.234+0000 7fc991237b80 1 Processor -- start 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x5556792c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555679050920 con 0x5556792a4860 2023-11-28T11:53:03.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556792b65d0 con 0x5556792a5170 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556792a1e70 con 0x5556792a2180 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 10 client.0 _ll_register_callbacks cb 0x555679072290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x5556792c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x5556792c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37718/0 (socket says 172.21.15.144:37718) 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 -- 172.21.15.144:0/848741286 learned_addr learned my addr 172.21.15.144:0/848741286 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97f59e700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 -- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 msgr2=0x5556792c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 -- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 msgr2=0x5556792c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 -- 172.21.15.144:0/848741286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc974009e20 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97f59e700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc97e59c700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x5556792c5880 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc974011830 tx=0x7fc9740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2d643c1d6c0bc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc95dffb700 1 -- 172.21.15.144:0/848741286 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974025070 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc95dffb700 1 -- 172.21.15.144:0/848741286 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc974004590 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.235+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55567919a190 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc95dffb700 1 -- 172.21.15.144:0/848741286 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc974003c90 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 10 client.5763 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555678ffade0 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc95dffb700 1 -- 172.21.15.144:0/848741286 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55567919a190 con 0x5556792a4860 2023-11-28T11:53:03.245 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc95dffb700 1 -- 172.21.15.144:0/848741286 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc97401e450 con 0x5556792a4860 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 10 client.5763 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 10 client.5763 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 20 client.5763 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55567929b330 0x5556792c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 4 client.5763 mds_command: new command op to 5535 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555679050f60 con 0x55567929b330 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55567929b330 0x5556792c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc95dffb700 10 client.5763 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.236+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55567929b330 0x5556792c4280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3938bcab12f2c86c server_cookie=7029e2f8db42c246 in_seq=0 out_seq=0 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc95dffb700 1 -- 172.21.15.144:0/848741286 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x555679050f60 con 0x55567929b330 2023-11-28T11:53:03.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc95dffb700 10 client.5763 handle_command_reply: tid=0 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 client.5763 shutdown 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 msgr2=0x5556792c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x5556792c5880 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc974011830 tx=0x7fc9740047c0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 shutdown_connections 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55567929b330 0x5556792c4280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc95dffb700 0 client.5763 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 --2- 172.21.15.144:0/848741286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x5556792c5880 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 >> 172.21.15.144:0/848741286 conn(0x555679277db0 msgr2=0x555679279280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 shutdown_connections 2023-11-28T11:53:03.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 1 -- 172.21.15.144:0/848741286 wait complete. 2023-11-28T11:53:03.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.237+0000 7fc991237b80 20 client.5763 trim_cache size 0 max 16384 2023-11-28T11:53:03.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.253+0000 7fc991237b80 1 Processor -- start 2023-11-28T11:53:03.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 -- start start 2023-11-28T11:53:03.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x555679279d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x55567927a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555679278cd0 con 0x5556792a4860 2023-11-28T11:53:03.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555679275210 con 0x5556792a5170 2023-11-28T11:53:03.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.254+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556792c7d40 con 0x5556792a2180 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x55567927a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x55567927a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37726/0 (socket says 172.21.15.144:37726) 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 -- 172.21.15.144:0/241917303 learned_addr learned my addr 172.21.15.144:0/241917303 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97f59e700 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x555679279d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 -- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 msgr2=0x555679279d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x555679279d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 -- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 msgr2=0x5556792b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 -- 172.21.15.144:0/241917303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555679193190 con 0x5556792a4860 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x555679279d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc97e59c700 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x55567927a2a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc9740093e0 tx=0x7fc974025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daafd62b317581d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc9657fa700 1 -- 172.21.15.144:0/241917303 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97401e960 con 0x5556792a4860 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc9657fa700 1 -- 172.21.15.144:0/241917303 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97401eb70 con 0x5556792a4860 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.255+0000 7fc9657fa700 1 -- 172.21.15.144:0/241917303 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97401e450 con 0x5556792a4860 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 -- 172.21.15.144:0/241917303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 msgr2=0x55567927a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x55567927a2a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc9740093e0 tx=0x7fc974025000 comp rx=0 tx=0).stop 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 -- 172.21.15.144:0/241917303 shutdown_connections 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a5170 0x5556792b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x555679279d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 --2- 172.21.15.144:0/241917303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a4860 0x55567927a2a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 -- 172.21.15.144:0/241917303 >> 172.21.15.144:0/241917303 conn(0x555679277db0 msgr2=0x5556792c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 -- 172.21.15.144:0/241917303 shutdown_connections 2023-11-28T11:53:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 -- 172.21.15.144:0/241917303 wait complete. 2023-11-28T11:53:03.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 -- start start 2023-11-28T11:53:03.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.256+0000 7fc991237b80 1 Processor -- start 2023-11-28T11:53:03.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x5556792a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 0x5556792b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556792c4fe0 con 0x5556792a5170 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556792b5660 con 0x5556792a4860 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55567929a030 con 0x5556792a2180 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc991237b80 10 client.0 _ll_register_callbacks cb 0x555679072290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 0x5556792b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 0x5556792b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37734/0 (socket says 172.21.15.144:37734) 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 -- 172.21.15.144:0/357894986 learned_addr learned my addr 172.21.15.144:0/357894986 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97e59c700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x5556792a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 -- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 msgr2=0x5556792a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 -- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 msgr2=0x5556792a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x5556792a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 -- 172.21.15.144:0/357894986 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555678ffade0 con 0x5556792a5170 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc97f59e700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 0x5556792b6af0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc978054100 tx=0x7fc978009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b996e39bdc196aa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.257+0000 7fc95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc95dffb700 1 -- 172.21.15.144:0/357894986 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780092e0 con 0x5556792a5170 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc95dffb700 1 -- 172.21.15.144:0/357894986 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc97805bb60 con 0x5556792a5170 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55567918a1b0 con 0x5556792a5170 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc95dffb700 1 -- 172.21.15.144:0/357894986 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc97805bd20 con 0x5556792a5170 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 10 client.5769 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555679021250 con 0x5556792a5170 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc95dffb700 1 -- 172.21.15.144:0/357894986 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55567918a1b0 con 0x5556792a5170 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc95dffb700 1 -- 172.21.15.144:0/357894986 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fc9780539f0 con 0x5556792a5170 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 10 client.5769 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 10 client.5769 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 20 client.5769 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x555679323f60 0x5556792b7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 4 client.5769 mds_command: new command op to 5535 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556791741c0 con 0x555679323f60 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x555679323f60 0x5556792b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc95dffb700 10 client.5769 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.258+0000 7fc97ed9d700 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x555679323f60 0x5556792b7030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0af81bb4221e9a9 server_cookie=9e8b3ec0955f54f6 in_seq=0 out_seq=0 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc95dffb700 1 -- 172.21.15.144:0/357894986 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5556791741c0 con 0x555679323f60 2023-11-28T11:53:03.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc95dffb700 10 client.5769 handle_command_reply: tid=0 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 client.5769 shutdown 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 msgr2=0x5556792b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 0x5556792b6af0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc978054100 tx=0x7fc978009a30 comp rx=0 tx=0).stop 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 shutdown_connections 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556792a5170 0x5556792b6af0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x555679323f60 0x5556792b7030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5556792a2180 0x5556792a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc95dffb700 0 client.5769 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 --2- 172.21.15.144:0/357894986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556792a4860 0x5556792a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 >> 172.21.15.144:0/357894986 conn(0x555679277db0 msgr2=0x555679279280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 shutdown_connections 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 1 -- 172.21.15.144:0/357894986 wait complete. 2023-11-28T11:53:03.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc991237b80 20 client.5769 trim_cache size 0 max 16384 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: "scrubs": { 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: "31574e77-d1b7-4f7e-9258-16fe2f4e3908": { 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: "path": "/", 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: "tag": "31574e77-d1b7-4f7e-9258-16fe2f4e3908", 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: "options": "recursive,repair" 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:03.269 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:03.270 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:03.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc96c0682f0 msgr2=0x7fc96c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc96c0682f0 0x7fc96c06a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc970000f60 tx=0x7fc970007ed0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 msgr2=0x7fc98014a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 0x7fc98014a4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc980155380 tx=0x7fc978004ad0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 shutdown_connections 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc980154e70 0x7fc98014a4b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc96c0682f0 0x7fc96c06a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc980071580 0x7fc98006cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 --2- 172.21.15.144:0/681857162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9800cdbd0 0x7fc9800cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.259+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 >> 172.21.15.144:0/681857162 conn(0x7fc9800d0000 msgr2=0x7fc9800afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.260+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 shutdown_connections 2023-11-28T11:53:03.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:03.260+0000 7fc9850ae700 1 -- 172.21.15.144:0/681857162 wait complete. 2023-11-28T11:53:03.273 INFO:tasks.cephfs.filesystem:scrub status for tag:31574e77-d1b7-4f7e-9258-16fe2f4e3908 - {'path': '/', 'tag': '31574e77-d1b7-4f7e-9258-16fe2f4e3908', 'options': 'recursive,repair'} 2023-11-28T11:53:04.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:04.734+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 7 ==== log(8 entries from seq 1 at 2023-11-28T11:53:03.086699+0000) v1 ==== 1638+0+0 (secure 0 0 0) 0x7f13f8083e40 con 0x7f14080cd370 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: cluster: 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: id: af726c13-c3f1-4279-9fca-12c6e4621b18 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: health: HEALTH_OK 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: services: 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: mgr: y(active, since 118s), standbys: x 2023-11-28T11:53:04.737 INFO:teuthology.orchestra.run.smithi144.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: osd: 8 osds: 8 up (since 113s), 8 in (since 2m) 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: data: 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: volumes: 1/1 healthy 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: pools: 4 pools, 105 pgs 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: objects: 24 objects, 579 KiB 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: usage: 72 MiB used, 720 GiB / 720 GiB avail 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: pgs: 105 active+clean 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: io: 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: client: 1.2 KiB/s wr, 0 op/s rd, 3 op/s wr 2023-11-28T11:53:04.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:03.086699+0000 mds.d [INF] scrub queued for path: / 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:03.086703+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:03.086708+0000 mds.d [INF] scrub summary: active paths [/] 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:03.087508+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:03.087518+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T11:53:04.739 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:03.087861+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:53:05.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:05.743+0000 7f13fe7fc700 1 -- 172.21.15.144:0/733611692 <== mon.0 v2:172.21.15.144:3300/0 8 ==== log(3 entries from seq 522 at 2023-11-28T11:53:04.738206+0000) v1 ==== 753+0+0 (secure 0 0 0) 0x7f13f8085ab0 con 0x7f14080cd370 2023-11-28T11:53:05.745 INFO:teuthology.orchestra.run.smithi144.stdout:2023-11-28T11:53:04.879562+0000 mds.d [INF] scrub summary: idle 2023-11-28T11:53:22.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:53:22.915 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:53:33.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:53:33.410 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.410+0000 7fbc470f6700 1 Processor -- start 2023-11-28T11:53:33.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.410+0000 7fbc470f6700 1 -- start start 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.410+0000 7fbc470f6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc400b3090 0x7fbc400cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.410+0000 7fbc470f6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc470f6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4015ad60 con 0x7fbc400cf070 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc470f6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc400cf4f0 con 0x7fbc400b3090 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc470f6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400adb90 con 0x7fbc400cb620 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc400b3090 0x7fbc400cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc400b3090 0x7fbc400cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37738/0 (socket says 172.21.15.144:37738) 2023-11-28T11:53:33.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc45693700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbc400cf070 0x7fbc400cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54452/0 (socket says 172.21.15.144:54452) 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 -- 172.21.15.144:0/3840386096 learned_addr learned my addr 172.21.15.144:0/3840386096 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/3840386096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 -- 172.21.15.144:0/3840386096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 msgr2=0x7fbc400ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 --2- 172.21.15.144:0/3840386096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 -- 172.21.15.144:0/3840386096 >> v1:172.21.15.144:6789/0 conn(0x7fbc400cf070 legacy=0x7fbc400cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 -- 172.21.15.144:0/3840386096 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400ccdd0 con 0x7fbc400b3090 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc3f7fe700 1 -- 172.21.15.144:0/3840386096 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629632098 0 0) 0x7fbc4015ad60 con 0x7fbc400cf070 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc44e92700 1 --2- 172.21.15.144:0/3840386096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400b3090 0x7fbc400cc4a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbc340098f0 tx=0x7fbc34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de93cea5f97c22c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc3f7fe700 1 -- 172.21.15.144:0/3840386096 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400b8a0 con 0x7fbc400b3090 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc3f7fe700 1 -- 172.21.15.144:0/3840386096 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3400ba00 con 0x7fbc400b3090 2023-11-28T11:53:33.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.411+0000 7fbc3f7fe700 1 -- 172.21.15.144:0/3840386096 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400bd90 con 0x7fbc400b3090 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 -- 172.21.15.144:0/3840386096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400b3090 msgr2=0x7fbc400cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 --2- 172.21.15.144:0/3840386096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400b3090 0x7fbc400cc4a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbc340098f0 tx=0x7fbc34009c00 comp rx=0 tx=0).stop 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 -- 172.21.15.144:0/3840386096 shutdown_connections 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 --2- 172.21.15.144:0/3840386096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400b3090 0x7fbc400cc4a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 --2- 172.21.15.144:0/3840386096 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 -- 172.21.15.144:0/3840386096 >> 172.21.15.144:0/3840386096 conn(0x7fbc4009a230 msgr2=0x7fbc4008afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 -- 172.21.15.144:0/3840386096 shutdown_connections 2023-11-28T11:53:33.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.412+0000 7fbc470f6700 1 -- 172.21.15.144:0/3840386096 wait complete. 2023-11-28T11:53:33.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.413+0000 7fbc470f6700 1 Processor -- start 2023-11-28T11:53:33.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.413+0000 7fbc470f6700 1 -- start start 2023-11-28T11:53:33.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc470f6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc400b3090 0x7fbc4014a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc470f6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc470f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 0x7fbc400b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.418 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc470f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b0c40 con 0x7fbc400cf070 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc470f6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b0030 con 0x7fbc400b3090 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc470f6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc400b0310 con 0x7fbc400cb620 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc45693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 0x7fbc400b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc3ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc3ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:49828/0 (socket says 172.21.15.144:49828) 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc3ffff700 1 -- 172.21.15.144:0/808663385 learned_addr learned my addr 172.21.15.144:0/808663385 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc45693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 0x7fbc400b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37748/0 (socket says 172.21.15.144:37748) 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.414+0000 7fbc44e92700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc400b3090 0x7fbc4014a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc45693700 1 -- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 msgr2=0x7fbc400b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc45693700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc45693700 1 -- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc400b3090 msgr2=0x7fbc4014a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc45693700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc400b3090 0x7fbc4014a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc45693700 1 -- 172.21.15.144:0/808663385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34009550 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc45693700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 0x7fbc400b5880 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cbe0 tx=0x7fbc3800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc3d7fa700 1 -- 172.21.15.144:0/808663385 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800aa40 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc3d7fa700 1 -- 172.21.15.144:0/808663385 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3800abc0 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc400cd390 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc3d7fa700 1 -- 172.21.15.144:0/808663385 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380070d0 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.415+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc400cd0b0 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.417+0000 7fbc3d7fa700 1 -- 172.21.15.144:0/808663385 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc3801a030 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.417+0000 7fbc3d7fa700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc240682f0 0x7fbc2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.417+0000 7fbc3d7fa700 1 -- 172.21.15.144:0/808663385 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbc380858a0 con 0x7fbc400cf070 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.417+0000 7fbc44e92700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc240682f0 0x7fbc2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.418+0000 7fbc44e92700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc240682f0 0x7fbc2406a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbc34014040 tx=0x7fbc34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.419+0000 7fbc5327fb80 1 Processor -- start 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.419+0000 7fbc5327fb80 1 -- start start 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77fee7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff0f180 0x55c77ff11c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff2d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77fdd1c10 con 0x55c77ff12170 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77fe403a0 con 0x55c77ff0f180 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c77fe398d0 con 0x55c77ff11860 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff2d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.427 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff2d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37756/0 (socket says 172.21.15.144:37756) 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 -- 172.21.15.144:0/710942080 learned_addr learned my addr 172.21.15.144:0/710942080 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff0f180 0x55c77ff11c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 -- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 msgr2=0x55c77fee7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77fee7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 -- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff0f180 msgr2=0x55c77ff11c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff0f180 0x55c77ff11c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 -- 172.21.15.144:0/710942080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c77fcbb630 con 0x55c77ff12170 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff0f180 0x55c77ff11c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc45693700 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff2d7c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbc38055780 tx=0x7fbc380557b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b503e9fb53eba64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc2a7fc700 1 -- 172.21.15.144:0/710942080 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3805db80 con 0x55c77ff12170 2023-11-28T11:53:33.429 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc2a7fc700 1 -- 172.21.15.144:0/710942080 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc38053f20 con 0x55c77ff12170 2023-11-28T11:53:33.429 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.420+0000 7fbc2a7fc700 1 -- 172.21.15.144:0/710942080 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38051d10 con 0x55c77ff12170 2023-11-28T11:53:33.429 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 -- 172.21.15.144:0/710942080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 msgr2=0x55c77ff2d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff2d7c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbc38055780 tx=0x7fbc380557b0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 -- 172.21.15.144:0/710942080 shutdown_connections 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff2d7c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77fee7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/710942080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff0f180 0x55c77ff11c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 -- 172.21.15.144:0/710942080 >> 172.21.15.144:0/710942080 conn(0x55c77fee4db0 msgr2=0x55c77fee5740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 -- 172.21.15.144:0/710942080 shutdown_connections 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.421+0000 7fbc5327fb80 1 -- 172.21.15.144:0/710942080 wait complete. 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.422+0000 7fbc5327fb80 1 -- start start 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.422+0000 7fbc5327fb80 1 Processor -- start 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77ff00390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77ff03720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77fcbd920 con 0x55c77ff0f180 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff04210 con 0x55c77ff12170 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff24b80 con 0x55c77ff11860 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 10 client.0 _ll_register_callbacks cb 0x55c77fcdf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77ff00390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77ff00390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37758/0 (socket says 172.21.15.144:37758) 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 -- 172.21.15.144:0/1331352221 learned_addr learned my addr 172.21.15.144:0/1331352221 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc45693700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77ff03720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 -- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 msgr2=0x55c77ff031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 -- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 msgr2=0x55c77ff03720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77ff03720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 -- 172.21.15.144:0/1331352221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc400cd0b0 con 0x55c77ff0f180 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc44e92700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77ff00390 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbc4006d870 tx=0x7fbc34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d923a95f83fe4353 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1331352221 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3402b070 con 0x55c77ff0f180 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1331352221 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbc340038b0 con 0x55c77ff0f180 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.423+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c77fe07190 con 0x55c77ff0f180 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1331352221 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3401f070 con 0x55c77ff0f180 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 10 client.5784 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55c77fc67de0 con 0x55c77ff0f180 2023-11-28T11:53:33.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1331352221 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c77fe07190 con 0x55c77ff0f180 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1331352221 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbc34003a70 con 0x55c77ff0f180 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 10 client.5784 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 10 client.5784 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 20 client.5784 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77ff01e80 0x55c77ff241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 4 client.5784 mds_command: new command op to 5535 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c77fcbdf60 con 0x55c77ff01e80 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77ff01e80 0x55c77ff241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc1f7fe700 10 client.5784 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.424+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77ff01e80 0x55c77ff241b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225c12930c1f3e09 server_cookie=b877aacb0f000a16 in_seq=0 out_seq=0 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1331352221 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c77fcbdf60 con 0x55c77ff01e80 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc1f7fe700 10 client.5784 handle_command_reply: tid=0 2023-11-28T11:53:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 client.5784 shutdown 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 msgr2=0x55c77ff00390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77ff00390 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbc4006d870 tx=0x7fbc34004750 comp rx=0 tx=0).stop 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 shutdown_connections 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77ff00390 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77ff01e80 0x55c77ff241b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77ff03720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc1f7fe700 0 client.5784 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1331352221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 >> 172.21.15.144:0/1331352221 conn(0x55c77fee4db0 msgr2=0x55c77fee79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 shutdown_connections 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.425+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1331352221 wait complete. 2023-11-28T11:53:33.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.426+0000 7fbc5327fb80 20 client.5784 trim_cache size 0 max 16384 2023-11-28T11:53:33.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.442+0000 7fbc5327fb80 1 Processor -- start 2023-11-28T11:53:33.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.442+0000 7fbc5327fb80 1 -- start start 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.442+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff0f180 0x55c77fee3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.442+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff11860 0x55c77ff348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff02b90 con 0x55c77ff12170 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff03770 con 0x55c77ff11860 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c77fee7d70 con 0x55c77ff0f180 2023-11-28T11:53:33.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff4e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37770/0 (socket says 172.21.15.144:37770) 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 -- 172.21.15.144:0/2990515807 learned_addr learned my addr 172.21.15.144:0/2990515807 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 -- 172.21.15.144:0/2990515807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff0f180 msgr2=0x55c77fee3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff0f180 0x55c77fee3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 -- 172.21.15.144:0/2990515807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff11860 msgr2=0x55c77ff348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff11860 0x55c77ff348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 -- 172.21.15.144:0/2990515807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c77fe00190 con 0x55c77ff12170 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc45693700 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff4e8b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbc38001d90 tx=0x7fbc38009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2669f8115493129d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc2affd700 1 -- 172.21.15.144:0/2990515807 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3805db80 con 0x55c77ff12170 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc2affd700 1 -- 172.21.15.144:0/2990515807 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc38061070 con 0x55c77ff12170 2023-11-28T11:53:33.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.443+0000 7fbc2affd700 1 -- 172.21.15.144:0/2990515807 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38051d10 con 0x55c77ff12170 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 -- 172.21.15.144:0/2990515807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 msgr2=0x55c77ff4e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff4e8b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbc38001d90 tx=0x7fbc38009070 comp rx=0 tx=0).stop 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 -- 172.21.15.144:0/2990515807 shutdown_connections 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff12170 0x55c77ff4e8b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff0f180 0x55c77fee3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/2990515807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff11860 0x55c77ff348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 -- 172.21.15.144:0/2990515807 >> 172.21.15.144:0/2990515807 conn(0x55c77fee4db0 msgr2=0x55c77fee1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 -- 172.21.15.144:0/2990515807 shutdown_connections 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 -- 172.21.15.144:0/2990515807 wait complete. 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 -- start start 2023-11-28T11:53:33.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.444+0000 7fbc5327fb80 1 Processor -- start 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77fee7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff95480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77fee04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff08710 con 0x55c77ff0f180 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff25fd0 con 0x55c77ff12170 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c77ff26210 con 0x55c77ff11860 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc5327fb80 10 client.0 _ll_register_callbacks cb 0x55c77fcdf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc44e92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77fee7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc44e92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77fee7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37772/0 (socket says 172.21.15.144:37772) 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc44e92700 1 -- 172.21.15.144:0/1263445268 learned_addr learned my addr 172.21.15.144:0/1263445268 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc45693700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77fee04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.445+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff95480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc44e92700 1 -- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 msgr2=0x55c77ff95480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc44e92700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff95480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc44e92700 1 -- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 msgr2=0x55c77fee04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc44e92700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77fee04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc44e92700 1 -- 172.21.15.144:0/1263445268 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c77fcbb630 con 0x55c77ff0f180 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc44e92700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77fee7cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c010 tx=0x7fbc340092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2536e9b35b76d036 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:33.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1263445268 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3401f070 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1263445268 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbc34030b90 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c77fdf71b0 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1263445268 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3400bd60 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 10 client.5790 fetch_fsmap learned FSMap version 24 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55c77fc8e250 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1263445268 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c77fdf71b0 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1263445268 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fbc34011420 con 0x55c77ff0f180 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 10 client.5790 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 10 client.5790 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 20 client.5790 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77fee09e0 0x55c77ff001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 4 client.5790 mds_command: new command op to 5535 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:53:33.453 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.446+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 --> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c77fde11c0 con 0x55c77fee09e0 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77fee09e0 0x55c77ff001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc1f7fe700 10 client.5790 ms_handle_connect on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc3ffff700 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77fee09e0 0x55c77ff001a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b8fab527c66e85 server_cookie=498e51c096fa7531 in_seq=0 out_seq=0 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc1f7fe700 1 -- 172.21.15.144:0/1263445268 <== mds.0 v2:172.21.15.177:6832/1698439070 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c77fde11c0 con 0x55c77fee09e0 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc1f7fe700 10 client.5790 handle_command_reply: tid=0 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 client.5790 shutdown 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 msgr2=0x55c77fee7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77fee7cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400c010 tx=0x7fbc340092c0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 shutdown_connections 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c77ff0f180 0x55c77fee7cd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.454 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:6832/1698439070,v1:172.21.15.177:6834/1698439070] conn(0x55c77fee09e0 0x55c77ff001a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c77ff12170 0x55c77fee04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 --2- 172.21.15.144:0/1263445268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c77ff11860 0x55c77ff95480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 >> 172.21.15.144:0/1263445268 conn(0x55c77fee4db0 msgr2=0x55c77ff25010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc1f7fe700 0 client.5790 ms_handle_reset on v2:172.21.15.177:6832/1698439070 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 shutdown_connections 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.447+0000 7fbc5327fb80 1 -- 172.21.15.144:0/1263445268 wait complete. 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc5327fb80 20 client.5790 trim_cache size 0 max 16384 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:33.455 INFO:teuthology.orchestra.run.smithi144.stdout: "status": "no active scrubs running", 2023-11-28T11:53:33.456 INFO:teuthology.orchestra.run.smithi144.stdout: "scrubs": {} 2023-11-28T11:53:33.456 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc240682f0 msgr2=0x7fbc2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc240682f0 0x7fbc2406a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbc34014040 tx=0x7fbc34008030 comp rx=0 tx=0).stop 2023-11-28T11:53:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 msgr2=0x7fbc400b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 0x7fbc400b5880 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cbe0 tx=0x7fbc3800cef0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 shutdown_connections 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc400cf070 0x7fbc400b5880 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc240682f0 0x7fbc2406a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc400b3090 0x7fbc4014a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 --2- 172.21.15.144:0/808663385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc400cb620 0x7fbc400b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 >> 172.21.15.144:0/808663385 conn(0x7fbc4009a230 msgr2=0x7fbc401e3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 shutdown_connections 2023-11-28T11:53:33.457 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.448+0000 7fbc470f6700 1 -- 172.21.15.144:0/808663385 wait complete. 2023-11-28T11:53:33.460 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:53:33.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:33.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.682+0000 7f0ef4f6c700 1 Processor -- start 2023-11-28T11:53:33.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.682+0000 7f0ef4f6c700 1 -- start start 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0ef4f6c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 0x7f0ef0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0ef4f6c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ef00cdc90 0x7f0ef00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0ef4f6c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef013ff00 con 0x7f0ef00b2e70 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0ef4f6c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef01404c0 con 0x7f0ef00cdc90 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0ef4f6c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef015aa80 con 0x7f0ef00cd8c0 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eeed9d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0ef00b2e70 0x7f0ef006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54496/0 (socket says 172.21.15.144:54496) 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eeed9d700 1 -- 172.21.15.144:0/536426891 learned_addr learned my addr 172.21.15.144:0/536426891 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/536426891 >> v2:172.21.15.177:3300/0 conn(0x7f0ef00cdc90 0x7f0ef00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540106237 0 0) 0x7f0ef013ff00 con 0x7f0ef00b2e70 2023-11-28T11:53:33.689 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ed80035b0 con 0x7f0ef00b2e70 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eee59c700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 0x7f0ef0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184052494 0 0) 0x7f0ed80035b0 con 0x7f0ef00b2e70 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 msgr2=0x7f0ef0140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 0x7f0ef0140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 >> v2:172.21.15.177:3300/0 conn(0x7f0ef00cdc90 msgr2=0x7f0ef00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 --2- 172.21.15.144:0/536426891 >> v2:172.21.15.177:3300/0 conn(0x7f0ef00cdc90 0x7f0ef00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00b3310 con 0x7f0ef00b2e70 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f0ee80019f0 con 0x7f0ef00b2e70 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 >> v1:172.21.15.144:6789/0 conn(0x7f0ef00b2e70 legacy=0x7f0ef006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 0x7f0ed8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed8003f80 0x7f0ed8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.690 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef01404c0 con 0x7f0ed8003f80 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef015aa80 con 0x7f0ed8004d90 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed800ba40 con 0x7f0ef00cd8c0 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.683+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed8003f80 0x7f0ed8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eee59c700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 0x7f0ed8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eeed9d700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eedd9b700 1 -- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 msgr2=0x7f0ed8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ef00cd8c0 0x7f0ed8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eedd9b700 1 -- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed8004d90 msgr2=0x7f0ed8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed8004d90 0x7f0ed8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eedd9b700 1 -- 172.21.15.144:0/536426891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80035b0 con 0x7f0ed8003f80 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed8003f80 0x7f0ed8004850 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009b30 tx=0x7f0ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32608f03f946bf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400e660 con 0x7f0ed8003f80 2023-11-28T11:53:33.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400e7c0 con 0x7f0ed8003f80 2023-11-28T11:53:33.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eed59a700 1 -- 172.21.15.144:0/536426891 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400eb30 con 0x7f0ed8003f80 2023-11-28T11:53:33.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/536426891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed8003f80 msgr2=0x7f0ed8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/536426891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ed8003f80 0x7f0ed8004850 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009b30 tx=0x7f0ed8005880 comp rx=0 tx=0).stop 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.684+0000 7f0eeed9d700 1 -- 172.21.15.144:0/536426891 reap_dead start 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.685+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/536426891 shutdown_connections 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.685+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/536426891 >> 172.21.15.144:0/536426891 conn(0x7f0ef006aed0 msgr2=0x7f0ef006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.685+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/536426891 shutdown_connections 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.685+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/536426891 wait complete. 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.686+0000 7f0ef4f6c700 1 Processor -- start 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.686+0000 7f0ef4f6c700 1 -- start start 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed8003f80 0x7f0ef00c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 0x7f0ef00c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ef0158a40 0x7f0ef0158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00cad00 con 0x7f0ef015c880 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef01593f0 con 0x7f0ef0158a40 2023-11-28T11:53:33.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef00c7f80 con 0x7f0ed8003f80 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 0x7f0ef00c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 0x7f0ef00c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37794/0 (socket says 172.21.15.144:37794) 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 -- 172.21.15.144:0/1114324366 learned_addr learned my addr 172.21.15.144:0/1114324366 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eee59c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed8003f80 0x7f0ef00c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eeed9d700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ef0158a40 0x7f0ef0158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 -- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed8003f80 msgr2=0x7f0ef00c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed8003f80 0x7f0ef00c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 -- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ef0158a40 msgr2=0x7f0ef0158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ef0158a40 0x7f0ef0158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 -- 172.21.15.144:0/1114324366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0140150 con 0x7f0ef015c880 2023-11-28T11:53:33.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0eedd9b700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 0x7f0ef00c3dc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8004770 tx=0x7f0ee40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400e660 con 0x7f0ef015c880 2023-11-28T11:53:33.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee4013cb0 con 0x7f0ef015c880 2023-11-28T11:53:33.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef00b35f0 con 0x7f0ef015c880 2023-11-28T11:53:33.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee401c690 con 0x7f0ef015c880 2023-11-28T11:53:33.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.687+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef00ce100 con 0x7f0ef015c880 2023-11-28T11:53:33.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.688+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef015a780 con 0x7f0ef015c880 2023-11-28T11:53:33.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.689+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ee4019030 con 0x7f0ef015c880 2023-11-28T11:53:33.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.689+0000 7f0edeffd700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0ed00682f0 0x7f0ed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:33.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.689+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0ee4087280 con 0x7f0ef015c880 2023-11-28T11:53:33.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.689+0000 7f0eee59c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0ed00682f0 0x7f0ed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:33.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.690+0000 7f0eee59c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0ed00682f0 0x7f0ed006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009b60 tx=0x7f0ee00091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:33.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.694+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0ee4053aa0 con 0x7f0ef015c880 2023-11-28T11:53:33.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.920+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ef00d08c0 con 0x7f0ef015c880 2023-11-28T11:53:33.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.921+0000 7f0edeffd700 1 -- 172.21.15.144:0/1114324366 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7f0ee40570c0 con 0x7f0ef015c880 2023-11-28T11:53:33.922 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:33.922 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:53:33.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0ed00682f0 msgr2=0x7f0ed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0ed00682f0 0x7f0ed006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009b60 tx=0x7f0ee00091c0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 msgr2=0x7f0ef00c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:33.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 0x7f0ef00c3dc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8004770 tx=0x7f0ee40047c0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 shutdown_connections 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0ed00682f0 0x7f0ed006a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ef0158a40 0x7f0ef0158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed8003f80 0x7f0ef00c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 --2- 172.21.15.144:0/1114324366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef015c880 0x7f0ef00c3dc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 >> 172.21.15.144:0/1114324366 conn(0x7f0ef006aed0 msgr2=0x7f0ef00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 shutdown_connections 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:33.923+0000 7f0ef4f6c700 1 -- 172.21.15.144:0/1114324366 wait complete. 2023-11-28T11:53:33.926 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:33.936 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:53:34.095 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2023-11-28T11:53:34.095 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:53:34.095 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:53:34.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.138+0000 7fb271269080 1 Processor -- start 2023-11-28T11:53:34.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.138+0000 7fb271269080 1 -- start start 2023-11-28T11:53:34.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb271269080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb271269080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae54231b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb271269080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb271269080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae54142fe0 con 0x55ae54235530 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb271269080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae54233c60 con 0x55ae54209b60 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb271269080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae54207b30 con 0x55ae542326f0 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb267b6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb267b6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542306c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37808/0 (socket says 172.21.15.144:37808) 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb267b6d700 1 -- 172.21.15.144:0/260863008 learned_addr learned my addr 172.21.15.144:0/260863008 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.139+0000 7fb26736c700 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb266b6b700 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae54231b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb267b6d700 1 -- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 msgr2=0x55ae54231b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb267b6d700 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae54231b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb267b6d700 1 -- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 msgr2=0x55ae542de3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb267b6d700 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542de3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb267b6d700 1 -- 172.21.15.144:0/260863008 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26000c3f0 con 0x55ae54235530 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb266b6b700 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae54231b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb267b6d700 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542306c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb26000cae0 tx=0x7fb260010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd817f888bf734 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb265b69700 1 -- 172.21.15.144:0/260863008 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000a7a0 con 0x55ae54235530 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb265b69700 1 -- 172.21.15.144:0/260863008 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26000a900 con 0x55ae54235530 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb265b69700 1 -- 172.21.15.144:0/260863008 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000f620 con 0x55ae54235530 2023-11-28T11:53:34.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 -- 172.21.15.144:0/260863008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 msgr2=0x55ae542306c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542306c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb26000cae0 tx=0x7fb260010000 comp rx=0 tx=0).stop 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 -- 172.21.15.144:0/260863008 shutdown_connections 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542306c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542de3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 --2- 172.21.15.144:0/260863008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae54231b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.140+0000 7fb271269080 1 -- 172.21.15.144:0/260863008 >> 172.21.15.144:0/260863008 conn(0x55ae54250330 msgr2=0x55ae54252c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.141+0000 7fb271269080 1 -- 172.21.15.144:0/260863008 shutdown_connections 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.141+0000 7fb271269080 1 -- 172.21.15.144:0/260863008 wait complete. 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.141+0000 7fb271269080 1 Processor -- start 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.141+0000 7fb271269080 1 -- start start 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb271269080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542d6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb271269080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae542d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb271269080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542e0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb271269080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae542d65b0 con 0x55ae54235530 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb271269080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5424c020 con 0x55ae54209b60 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb271269080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae5424c300 con 0x55ae542326f0 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb267b6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb267b6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542e0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37814/0 (socket says 172.21.15.144:37814) 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb267b6d700 1 -- 172.21.15.144:0/1691178225 learned_addr learned my addr 172.21.15.144:0/1691178225 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb26736c700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.142+0000 7fb266b6b700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae542d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb267b6d700 1 -- 172.21.15.144:0/1691178225 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 msgr2=0x55ae542d6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb267b6d700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55ae542326f0 0x55ae542d6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb267b6d700 1 -- 172.21.15.144:0/1691178225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 msgr2=0x55ae542d6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb267b6d700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae54209b60 0x55ae542d6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb267b6d700 1 -- 172.21.15.144:0/1691178225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae541754f0 con 0x55ae54235530 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb267b6d700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542e0820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb260025040 tx=0x7fb260003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb264b67700 1 -- 172.21.15.144:0/1691178225 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000ae70 con 0x55ae54235530 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb264b67700 1 -- 172.21.15.144:0/1691178225 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26002d420 con 0x55ae54235530 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ae5415b120 con 0x55ae54235530 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb264b67700 1 -- 172.21.15.144:0/1691178225 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26002d7d0 con 0x55ae54235530 2023-11-28T11:53:34.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.143+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae5414e830 con 0x55ae54235530 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.144+0000 7fb264b67700 1 -- 172.21.15.144:0/1691178225 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb260021070 con 0x55ae54235530 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb264b67700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb254066c10 0x7fb2540690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb264b67700 1 -- 172.21.15.144:0/1691178225 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb260089b90 con 0x55ae54235530 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb26736c700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb254066c10 0x7fb2540690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb271269080 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x55ae5424b7f0 0x55ae5424d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48) v8 -- 0x55ae5424d600 con 0x55ae5424b7f0 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb266b6b700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x55ae5424b7f0 0x55ae5424d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb26736c700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb254066c10 0x7fb2540690c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55ae54209550 tx=0x7fb25c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.151 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.145+0000 7fb266b6b700 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x55ae5424b7f0 0x55ae5424d0c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb266b6b700 1 -- 172.21.15.144:0/1691178225 <== osd.1 v2:172.21.15.144:6808/103557 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fb258010780 con 0x55ae5424b7f0 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x55ae5424b7f0 msgr2=0x55ae5424d0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x55ae5424b7f0 0x55ae5424d0c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb254066c10 msgr2=0x7fb2540690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb254066c10 0x7fb2540690c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55ae54209550 tx=0x7fb25c00d040 comp rx=0 tx=0).stop 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 msgr2=0x55ae542e0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 --2- 172.21.15.144:0/1691178225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae54235530 0x55ae542e0820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb260025040 tx=0x7fb260003330 comp rx=0 tx=0).stop 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb267b6d700 1 -- 172.21.15.144:0/1691178225 reap_dead start 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 shutdown_connections 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 >> 172.21.15.144:0/1691178225 conn(0x55ae54250330 msgr2=0x55ae54232ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 shutdown_connections 2023-11-28T11:53:34.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.146+0000 7fb271269080 1 -- 172.21.15.144:0/1691178225 wait complete. 2023-11-28T11:53:34.153 INFO:teuthology.orchestra.run.smithi144.stdout:Yalphaparent_a 2023-11-28T11:53:34.154 DEBUG:teuthology.orchestra.run.smithi144:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 1099511627777, 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout: "ancestors": [ 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout: "dirino": 1099511627776, 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout: "dname": "alpha", 2023-11-28T11:53:34.248 INFO:teuthology.orchestra.run.smithi144.stdout: "version": 2 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: "dirino": 1, 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: "dname": "parent_a", 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: "version": 4 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: "pool": 8, 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: "old_pools": [] 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:34.249 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:34.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:53:34.251 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:53:34.328 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:53:34.374 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:53:34.375 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:53:34.375 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:34.392 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:53:34.393 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:34.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:34.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.608+0000 7f9ea5dbb700 1 Processor -- start 2023-11-28T11:53:34.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.608+0000 7f9ea5dbb700 1 -- start start 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9ea5dbb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9ea0071040 0x7f9ea0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9ea5dbb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ea0160540 0x7f9ea0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9ea5dbb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea0071950 con 0x7f9ea006dee0 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9ea5dbb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea015d060 con 0x7f9ea0160540 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9ea5dbb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea015d340 con 0x7f9ea0071040 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9e9ffff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ea0160540 0x7f9ea0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9e9effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9ea006dee0 0x7f9ea006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54534/0 (socket says 172.21.15.144:54534) 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.609+0000 7f9e9effd700 1 -- 172.21.15.144:0/2953385185 learned_addr learned my addr 172.21.15.144:0/2953385185 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9f7fe700 1 --2- 172.21.15.144:0/2953385185 >> v2:172.21.15.177:3300/0 conn(0x7f9ea0071040 0x7f9ea0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9ffff700 1 -- 172.21.15.144:0/2953385185 >> v2:172.21.15.177:3300/0 conn(0x7f9ea0071040 msgr2=0x7f9ea0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9ffff700 1 --2- 172.21.15.144:0/2953385185 >> v2:172.21.15.177:3300/0 conn(0x7f9ea0071040 0x7f9ea0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9ffff700 1 -- 172.21.15.144:0/2953385185 >> v1:172.21.15.144:6789/0 conn(0x7f9ea006dee0 legacy=0x7f9ea006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9ffff700 1 -- 172.21.15.144:0/2953385185 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea015d620 con 0x7f9ea0160540 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9e7fc700 1 -- 172.21.15.144:0/2953385185 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094663978 0 0) 0x7f9ea0071950 con 0x7f9ea006dee0 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9f7fe700 1 --2- 172.21.15.144:0/2953385185 >> v2:172.21.15.177:3300/0 conn(0x7f9ea0071040 0x7f9ea0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9ffff700 1 --2- 172.21.15.144:0/2953385185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea0160540 0x7f9ea0160980 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00d0f50 tx=0x7f9e9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593def167d410224 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9e7fc700 1 -- 172.21.15.144:0/2953385185 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800e070 con 0x7f9ea0160540 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9e7fc700 1 -- 172.21.15.144:0/2953385185 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9800bbd0 con 0x7f9ea0160540 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9e9e7fc700 1 -- 172.21.15.144:0/2953385185 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98012430 con 0x7f9ea0160540 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/2953385185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea0160540 msgr2=0x7f9ea0160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.610+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/2953385185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea0160540 0x7f9ea0160980 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00d0f50 tx=0x7f9e9800b330 comp rx=0 tx=0).stop 2023-11-28T11:53:34.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.611+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/2953385185 shutdown_connections 2023-11-28T11:53:34.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.611+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/2953385185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea0160540 0x7f9ea0160980 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.611+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/2953385185 >> v2:172.21.15.177:3300/0 conn(0x7f9ea0071040 0x7f9ea0071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.611+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/2953385185 >> 172.21.15.144:0/2953385185 conn(0x7f9ea00d0000 msgr2=0x7f9ea00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:34.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.611+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/2953385185 shutdown_connections 2023-11-28T11:53:34.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.611+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/2953385185 wait complete. 2023-11-28T11:53:34.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.612+0000 7f9ea5dbb700 1 Processor -- start 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.612+0000 7f9ea5dbb700 1 -- start start 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 0x7f9ea01f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0071040 0x7f9ea00c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0160540 0x7f9ea015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01f5a80 con 0x7f9ea006dee0 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01f69a0 con 0x7f9ea0160540 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00c63e0 con 0x7f9ea0071040 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 0x7f9ea01f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 0x7f9ea01f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37826/0 (socket says 172.21.15.144:37826) 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 -- 172.21.15.144:0/1646587748 learned_addr learned my addr 172.21.15.144:0/1646587748 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:34.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 -- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0071040 msgr2=0x7f9ea00c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9ffff700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0160540 0x7f9ea015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0071040 0x7f9ea00c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 -- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0160540 msgr2=0x7f9ea015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0160540 0x7f9ea015ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 -- 172.21.15.144:0/1646587748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98009a70 con 0x7f9ea006dee0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e9f7fe700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 0x7f9ea01f6460 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800e810 tx=0x7f9e8800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800c020 con 0x7f9ea006dee0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e88004250 con 0x7f9ea006dee0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea015ccd0 con 0x7f9ea006dee0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e880036a0 con 0x7f9ea006dee0 2023-11-28T11:53:34.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.613+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea015d900 con 0x7f9ea006dee0 2023-11-28T11:53:34.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.614+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea006a270 con 0x7f9ea006dee0 2023-11-28T11:53:34.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.615+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e88003800 con 0x7f9ea006dee0 2023-11-28T11:53:34.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.615+0000 7f9e97fff700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e800682f0 0x7f9e8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.616+0000 7f9e9effd700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e800682f0 0x7f9e8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.616+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9e880861e0 con 0x7f9ea006dee0 2023-11-28T11:53:34.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.616+0000 7f9e9effd700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e800682f0 0x7f9e8006a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9ea01f59e0 tx=0x7f9e900079f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.619+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e88052aa0 con 0x7f9ea006dee0 2023-11-28T11:53:34.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.806+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ea01f62b0 con 0x7f9ea006dee0 2023-11-28T11:53:34.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.807+0000 7f9e97fff700 1 -- 172.21.15.144:0/1646587748 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18009 (secure 0 0 0) 0x7f9e8800f040 con 0x7f9ea006dee0 2023-11-28T11:53:34.808 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:34.808 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":48,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:05.732387+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e800682f0 msgr2=0x7f9e8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e800682f0 0x7f9e8006a7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9ea01f59e0 tx=0x7f9e900079f0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 msgr2=0x7f9ea01f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 0x7f9ea01f6460 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800e810 tx=0x7f9e8800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 shutdown_connections 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea006dee0 0x7f9ea01f6460 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e800682f0 0x7f9e8006a7a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0160540 0x7f9ea015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 --2- 172.21.15.144:0/1646587748 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0071040 0x7f9ea00c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 >> 172.21.15.144:0/1646587748 conn(0x7f9ea00d0000 msgr2=0x7f9ea00cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 shutdown_connections 2023-11-28T11:53:34.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.810+0000 7f9ea5dbb700 1 -- 172.21.15.144:0/1646587748 wait complete. 2023-11-28T11:53:34.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:52:48.228399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:52:49.180034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:53:34.823 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:53:34.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.945+0000 7fe8f366e700 1 Processor -- start 2023-11-28T11:53:34.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.946+0000 7fe8f366e700 1 -- start start 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.946+0000 7fe8f366e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe8ec06e9f0 0x7fe8ec06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.946+0000 7fe8f366e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06d0a0 0x7fe8ec0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.946+0000 7fe8f366e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec06ae60 con 0x7fe8ec06b830 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.946+0000 7fe8f366e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec06a270 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.946+0000 7fe8f366e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec06a530 con 0x7fe8ec06d0a0 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f140a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe8ec06e9f0 0x7fe8ec06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f0c09700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe8ec06b830 0x7fe8ec06c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54548/0 (socket says 172.21.15.144:54548) 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f0c09700 1 -- 172.21.15.144:0/4174366135 learned_addr learned my addr 172.21.15.144:0/4174366135 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:34.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f1c0b700 1 --2- 172.21.15.144:0/4174366135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06d0a0 0x7fe8ec0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f140a700 1 -- 172.21.15.144:0/4174366135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06d0a0 msgr2=0x7fe8ec0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f140a700 1 --2- 172.21.15.144:0/4174366135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06d0a0 0x7fe8ec0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f140a700 1 -- 172.21.15.144:0/4174366135 >> v1:172.21.15.144:6789/0 conn(0x7fe8ec06b830 legacy=0x7fe8ec06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f140a700 1 -- 172.21.15.144:0/4174366135 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec06a810 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8e3fff700 1 -- 172.21.15.144:0/4174366135 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872113604 0 0) 0x7fe8ec06ae60 con 0x7fe8ec06b830 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8f140a700 1 --2- 172.21.15.144:0/4174366135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec06b2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009840 tx=0x7fe8dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb977a943d8740a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8e3fff700 1 -- 172.21.15.144:0/4174366135 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc011830 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8e3fff700 1 -- 172.21.15.144:0/4174366135 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc011990 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.947+0000 7fe8e3fff700 1 -- 172.21.15.144:0/4174366135 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc011d20 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 -- 172.21.15.144:0/4174366135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 msgr2=0x7fe8ec06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 --2- 172.21.15.144:0/4174366135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec06b2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009840 tx=0x7fe8dc009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:34.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 -- 172.21.15.144:0/4174366135 shutdown_connections 2023-11-28T11:53:34.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 --2- 172.21.15.144:0/4174366135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06d0a0 0x7fe8ec0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 --2- 172.21.15.144:0/4174366135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec06b2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 -- 172.21.15.144:0/4174366135 >> 172.21.15.144:0/4174366135 conn(0x7fe8ec0b12b0 msgr2=0x7fe8ec0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:34.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 -- 172.21.15.144:0/4174366135 shutdown_connections 2023-11-28T11:53:34.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.948+0000 7fe8f366e700 1 -- 172.21.15.144:0/4174366135 wait complete. 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.949+0000 7fe8f366e700 1 Processor -- start 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.949+0000 7fe8f366e700 1 -- start start 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06b830 0x7fe8ec1f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec06d0a0 0x7fe8ec0c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec1e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c33c0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c7a10 con 0x7fe8ec06d0a0 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c82b0 con 0x7fe8ec06b830 2023-11-28T11:53:34.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec1e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec1e4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37844/0 (socket says 172.21.15.144:37844) 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 -- 172.21.15.144:0/2281982143 learned_addr learned my addr 172.21.15.144:0/2281982143 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f0c09700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec06d0a0 0x7fe8ec0c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 -- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06b830 msgr2=0x7fe8ec1f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f140a700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06b830 0x7fe8ec1f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06b830 0x7fe8ec1f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 -- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec06d0a0 msgr2=0x7fe8ec0c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec06d0a0 0x7fe8ec0c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 -- 172.21.15.144:0/2281982143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0094a0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f0c09700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec06d0a0 0x7fe8ec0c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f1c0b700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec1e4cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009e30 tx=0x7fe8e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4009ca0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e400f040 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec06edc0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.950+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4014480 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.951+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec06aaf0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.951+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec06ef50 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.952+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe8e40145e0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.952+0000 7fe8e1ffb700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8d00682f0 0x7fe8d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.952+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe8e4086ad0 con 0x7fe8ec06e9f0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.952+0000 7fe8f140a700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8d00682f0 0x7fe8d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:34.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.953+0000 7fe8f140a700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8d00682f0 0x7fe8d006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0090b0 tx=0x7fe8dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:34.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:34.957+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe8e40532f0 con 0x7fe8ec06e9f0 2023-11-28T11:53:35.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.137+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe8ec0c83f0 con 0x7fe8ec06e9f0 2023-11-28T11:53:35.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.138+0000 7fe8e1ffb700 1 -- 172.21.15.144:0/2281982143 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe8e40180a0 con 0x7fe8ec06e9f0 2023-11-28T11:53:35.139 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8d00682f0 msgr2=0x7fe8d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8d00682f0 0x7fe8d006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0090b0 tx=0x7fe8dc008030 comp rx=0 tx=0).stop 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 msgr2=0x7fe8ec1e4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec1e4cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009e30 tx=0x7fe8e400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 shutdown_connections 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8ec06e9f0 0x7fe8ec1e4cd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe8d00682f0 0x7fe8d006a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8ec06b830 0x7fe8ec1f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 --2- 172.21.15.144:0/2281982143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec06d0a0 0x7fe8ec0c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 >> 172.21.15.144:0/2281982143 conn(0x7fe8ec0b12b0 msgr2=0x7fe8ec065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 shutdown_connections 2023-11-28T11:53:35.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.140+0000 7fe8f366e700 1 -- 172.21.15.144:0/2281982143 wait complete. 2023-11-28T11:53:35.152 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:53:35.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 Processor -- start 2023-11-28T11:53:35.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 -- start start 2023-11-28T11:53:35.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71200cd8c0 0x7f7120140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f71200b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f71200b2e70 0x7f712006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712013ff00 con 0x7f71200cd8c0 2023-11-28T11:53:35.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71201404c0 con 0x7f71200b2e70 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.275+0000 7f7126e2b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712015aa80 con 0x7f71200cdc90 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71200cd8c0 0x7f7120140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71200cd8c0 0x7f7120140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37848/0 (socket says 172.21.15.144:37848) 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 -- 172.21.15.144:0/1216470132 learned_addr learned my addr 172.21.15.144:0/1216470132 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f711ffff700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f71200b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f71253c8700 1 --2- 172.21.15.144:0/1216470132 >> v2:172.21.15.177:3300/0 conn(0x7f71200b2e70 0x7f712006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 -- 172.21.15.144:0/1216470132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 msgr2=0x7f71200b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f71200b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 -- 172.21.15.144:0/1216470132 >> v2:172.21.15.177:3300/0 conn(0x7f71200b2e70 msgr2=0x7f712006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 --2- 172.21.15.144:0/1216470132 >> v2:172.21.15.177:3300/0 conn(0x7f71200b2e70 0x7f712006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 -- 172.21.15.144:0/1216470132 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200b3310 con 0x7f71200cd8c0 2023-11-28T11:53:35.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f711ffff700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f71200b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f7124bc7700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200cd8c0 0x7f7120140950 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f71201444c0 tx=0x7f7114009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c78427c24d06a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f711f7fe700 1 -- 172.21.15.144:0/1216470132 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114005e30 con 0x7f71200cd8c0 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f711f7fe700 1 -- 172.21.15.144:0/1216470132 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7114004030 con 0x7f71200cd8c0 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.276+0000 7f711f7fe700 1 -- 172.21.15.144:0/1216470132 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140043e0 con 0x7f71200cd8c0 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 -- 172.21.15.144:0/1216470132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200cd8c0 msgr2=0x7f7120140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200cd8c0 0x7f7120140950 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f71201444c0 tx=0x7f7114009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 -- 172.21.15.144:0/1216470132 shutdown_connections 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1216470132 >> v2:172.21.15.177:3300/0 conn(0x7f71200b2e70 0x7f712006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200cd8c0 0x7f7120140950 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1216470132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f71200b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 -- 172.21.15.144:0/1216470132 >> 172.21.15.144:0/1216470132 conn(0x7f712006aed0 msgr2=0x7f712006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:35.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 -- 172.21.15.144:0/1216470132 shutdown_connections 2023-11-28T11:53:35.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.277+0000 7f7126e2b700 1 -- 172.21.15.144:0/1216470132 wait complete. 2023-11-28T11:53:35.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.278+0000 7f7126e2b700 1 Processor -- start 2023-11-28T11:53:35.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.278+0000 7f7126e2b700 1 -- start start 2023-11-28T11:53:35.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.279+0000 7f7126e2b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 0x7f71200b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.279+0000 7f7126e2b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71200cd8c0 0x7f7120157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.279+0000 7f7126e2b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f7120156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.279+0000 7f7126e2b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120141d10 con 0x7f71200b2e70 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.279+0000 7f7126e2b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120159410 con 0x7f71200cd8c0 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.279+0000 7f7126e2b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7120157550 con 0x7f71200cdc90 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 0x7f71200b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 0x7f71200b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37854/0 (socket says 172.21.15.144:37854) 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 -- 172.21.15.144:0/1639987390 learned_addr learned my addr 172.21.15.144:0/1639987390 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f71253c8700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f7120156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f711ffff700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71200cd8c0 0x7f7120157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 -- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 msgr2=0x7f7120156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f7120156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 -- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71200cd8c0 msgr2=0x7f7120157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71200cd8c0 0x7f7120157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 -- 172.21.15.144:0/1639987390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114009500 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f71253c8700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f7120156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f711ffff700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71200cd8c0 0x7f7120157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7124bc7700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 0x7f71200b62c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f71200b3aa0 tx=0x7f7114004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114003d90 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711401c420 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7120140150 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711401c7b0 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.280+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71200b35f0 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.281+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71200ce100 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.282+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f711402c070 con 0x7f71200b2e70 2023-11-28T11:53:35.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.282+0000 7f711d7fa700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f71040682f0 0x7f710406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.282+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7114086d60 con 0x7f71200b2e70 2023-11-28T11:53:35.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.282+0000 7f711ffff700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f71040682f0 0x7f710406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.282+0000 7f711ffff700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f71040682f0 0x7f710406a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f712013dd90 tx=0x7f710c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.286+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7114053580 con 0x7f71200b2e70 2023-11-28T11:53:35.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.469+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7120068690 con 0x7f71200b2e70 2023-11-28T11:53:35.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.469+0000 7f711d7fa700 1 -- 172.21.15.144:0/1639987390 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7114016020 con 0x7f71200b2e70 2023-11-28T11:53:35.470 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:53:35.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f71040682f0 msgr2=0x7f710406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f71040682f0 0x7f710406a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f712013dd90 tx=0x7f710c008040 comp rx=0 tx=0).stop 2023-11-28T11:53:35.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 msgr2=0x7f71200b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.472 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 0x7f71200b62c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f71200b3aa0 tx=0x7f7114004750 comp rx=0 tx=0).stop 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 shutdown_connections 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71200b2e70 0x7f71200b62c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f71040682f0 0x7f710406a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71200cdc90 0x7f7120156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 --2- 172.21.15.144:0/1639987390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71200cd8c0 0x7f7120157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 >> 172.21.15.144:0/1639987390 conn(0x7f712006aed0 msgr2=0x7f71200b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 shutdown_connections 2023-11-28T11:53:35.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.471+0000 7f7126e2b700 1 -- 172.21.15.144:0/1639987390 wait complete. 2023-11-28T11:53:35.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:53:35.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.607+0000 7f9bb77c4700 1 Processor -- start 2023-11-28T11:53:35.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.607+0000 7f9bb77c4700 1 -- start start 2023-11-28T11:53:35.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb77c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9bb0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb77c4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb0071950 con 0x7f9bb0160540 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb77c4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb015d060 con 0x7f9bb006dee0 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb77c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb015d340 con 0x7f9bb0071040 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb5d61700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9bb0160540 0x7f9bb01609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54578/0 (socket says 172.21.15.144:54578) 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb5d61700 1 -- 172.21.15.144:0/4243661711 learned_addr learned my addr 172.21.15.144:0/4243661711 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9bb5560700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9bb0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879049813 0 0) 0x7f9bb0071950 con 0x7f9bb0160540 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9c0035b0 con 0x7f9bb0160540 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2012504929 0 0) 0x7f9b9c0035b0 con 0x7f9bb0160540 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 msgr2=0x7f9bb0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9ba7fff700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9bb0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.608+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 >> v1:172.21.15.177:6789/0 conn(0x7f9bb006dee0 legacy=0x7f9bb006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb015d620 con 0x7f9bb0160540 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f9ba80019f0 con 0x7f9bb0160540 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 >> v1:172.21.15.144:6789/0 conn(0x7f9bb0160540 legacy=0x7f9bb01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c003a30 0x7f9b9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9b9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c004d90 0x7f9b9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb015d340 con 0x7f9b9c003a30 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00bb20 con 0x7f9b9c004d90 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00bce0 con 0x7f9bb0071040 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c003a30 0x7f9b9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb4d5f700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9b9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5d61700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c004d90 0x7f9b9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 -- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 msgr2=0x7f9b9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9b9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 -- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c004d90 msgr2=0x7f9b9c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c004d90 0x7f9b9c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 -- 172.21.15.144:0/4243661711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0035b0 con 0x7f9b9c003a30 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb4d5f700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bb0071040 0x7f9b9c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5d61700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c004d90 0x7f9b9c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9bb5560700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c003a30 0x7f9b9c003e00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009b60 tx=0x7f9b9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56bf5be2ff84bda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00e6c0 con 0x7f9b9c003a30 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac00e820 con 0x7f9b9c003a30 2023-11-28T11:53:35.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.609+0000 7f9ba7fff700 1 -- 172.21.15.144:0/4243661711 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00eb90 con 0x7f9b9c003a30 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb77c4700 1 -- 172.21.15.144:0/4243661711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c003a30 msgr2=0x7f9b9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/4243661711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c003a30 0x7f9b9c003e00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009b60 tx=0x7f9b9c005880 comp rx=0 tx=0).stop 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb5d61700 1 -- 172.21.15.144:0/4243661711 reap_dead start 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb77c4700 1 -- 172.21.15.144:0/4243661711 shutdown_connections 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb77c4700 1 -- 172.21.15.144:0/4243661711 >> 172.21.15.144:0/4243661711 conn(0x7f9bb00d0000 msgr2=0x7f9bb00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb77c4700 1 -- 172.21.15.144:0/4243661711 shutdown_connections 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.610+0000 7f9bb77c4700 1 -- 172.21.15.144:0/4243661711 wait complete. 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.611+0000 7f9bb77c4700 1 Processor -- start 2023-11-28T11:53:35.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.611+0000 7f9bb77c4700 1 -- start start 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb77c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c003a30 0x7f9bb00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb77c4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 0x7f9bb00c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb77c4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bb0156690 0x7f9bb00b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb77c4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb00b3370 con 0x7f9bb00ae980 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb77c4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb015a940 con 0x7f9bb0156690 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb77c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb00aef40 con 0x7f9b9c003a30 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 0x7f9bb00c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 0x7f9bb00c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37876/0 (socket says 172.21.15.144:37876) 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 -- 172.21.15.144:0/2443071311 learned_addr learned my addr 172.21.15.144:0/2443071311 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:35.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb5560700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c003a30 0x7f9bb00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb5d61700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bb0156690 0x7f9bb00b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 -- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c003a30 msgr2=0x7f9bb00b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c003a30 0x7f9bb00b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 -- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bb0156690 msgr2=0x7f9bb00b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bb0156690 0x7f9bb00b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 -- 172.21.15.144:0/2443071311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb015ccd0 con 0x7f9bb00ae980 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb5d61700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bb0156690 0x7f9bb00b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9bb4d5f700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 0x7f9bb00c3a40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e830 tx=0x7f9ba000eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.612+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00052e0 con 0x7f9bb00ae980 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.613+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba0005440 con 0x7f9bb00ae980 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.613+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb015d900 con 0x7f9bb00ae980 2023-11-28T11:53:35.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.613+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00038a0 con 0x7f9bb00ae980 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.613+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb006a270 con 0x7f9bb00ae980 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.614+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb00720a0 con 0x7f9bb00ae980 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.614+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ba0003a00 con 0x7f9bb00ae980 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.614+0000 7f9ba5ffb700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b940682f0 0x7f9b9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.614+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9ba0085c40 con 0x7f9bb00ae980 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.614+0000 7f9bb5560700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b940682f0 0x7f9b9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.615+0000 7f9bb5560700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b940682f0 0x7f9b9406a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0090b0 tx=0x7f9bac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.619+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ba00523e0 con 0x7f9bb00ae980 2023-11-28T11:53:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.801+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9bb00cd610 con 0x7f9bb00ae980 2023-11-28T11:53:35.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.802+0000 7f9ba5ffb700 1 -- 172.21.15.144:0/2443071311 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ba0055a00 con 0x7f9bb00ae980 2023-11-28T11:53:35.802 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b940682f0 msgr2=0x7f9b9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b940682f0 0x7f9b9406a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0090b0 tx=0x7f9bac008030 comp rx=0 tx=0).stop 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 msgr2=0x7f9bb00c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 0x7f9bb00c3a40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e830 tx=0x7f9ba000eb40 comp rx=0 tx=0).stop 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 shutdown_connections 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b940682f0 0x7f9b9406a7a0 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0090b0 tx=0x7f9bac008030 comp rx=0 tx=0).stop 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bb0156690 0x7f9bb00b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c003a30 0x7f9bb00b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 --2- 172.21.15.144:0/2443071311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bb00ae980 0x7f9bb00c3a40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 >> 172.21.15.144:0/2443071311 conn(0x7f9bb00d0000 msgr2=0x7f9bb015e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:35.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 shutdown_connections 2023-11-28T11:53:35.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.804+0000 7f9bb77c4700 1 -- 172.21.15.144:0/2443071311 wait complete. 2023-11-28T11:53:35.817 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:53:35.939 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.939+0000 7ff3c10a8700 1 Processor -- start 2023-11-28T11:53:35.940 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.939+0000 7ff3c10a8700 1 -- start start 2023-11-28T11:53:35.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.939+0000 7ff3c10a8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff3bc140950 0x7ff3bc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.939+0000 7ff3c10a8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc13ff00 con 0x7ff3bc0b2e70 2023-11-28T11:53:35.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.939+0000 7ff3c10a8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc1404c0 con 0x7ff3bc0cd8c0 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.939+0000 7ff3c10a8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc140dc0 con 0x7ff3bc140950 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3bb59e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff3bc0b2e70 0x7ff3bc06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54598/0 (socket says 172.21.15.144:54598) 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3bb59e700 1 -- 172.21.15.144:0/1712173059 learned_addr learned my addr 172.21.15.144:0/1712173059 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3ba59c700 1 --2- 172.21.15.144:0/1712173059 >> v2:172.21.15.177:3300/0 conn(0x7ff3bc140950 0x7ff3bc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292625567 0 0) 0x7ff3bc13ff00 con 0x7ff3bc0b2e70 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a40035b0 con 0x7ff3bc0b2e70 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519771536 0 0) 0x7ff3bc1404c0 con 0x7ff3bc0cd8c0 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3bc13ff00 con 0x7ff3bc0cd8c0 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 477883180 0 0) 0x7ff3a40035b0 con 0x7ff3bc0b2e70 2023-11-28T11:53:35.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 >> v2:172.21.15.177:3300/0 conn(0x7ff3bc140950 msgr2=0x7ff3bc0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 --2- 172.21.15.144:0/1712173059 >> v2:172.21.15.177:3300/0 conn(0x7ff3bc140950 0x7ff3bc0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 >> v1:172.21.15.177:6789/0 conn(0x7ff3bc0cd8c0 legacy=0x7ff3bc0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc15aa80 con 0x7ff3bc0b2e70 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7ff3b40019f0 con 0x7ff3bc0b2e70 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3ba59c700 1 --2- 172.21.15.144:0/1712173059 >> v2:172.21.15.177:3300/0 conn(0x7ff3bc140950 0x7ff3bc0b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 >> v1:172.21.15.144:6789/0 conn(0x7ff3bc0b2e70 legacy=0x7ff3bc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a4003b90 0x7ff3a4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4004850 0x7ff3a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3a4005200 0x7ff3a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc140dc0 con 0x7ff3a4003b90 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a400c110 con 0x7ff3a4004850 2023-11-28T11:53:35.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a400c2d0 con 0x7ff3a4005200 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a4003b90 0x7ff3a4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3ba59c700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4004850 0x7ff3a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.940+0000 7ff3bb59e700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3a4005200 0x7ff3a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bad9d700 1 -- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3a4005200 msgr2=0x7ff3a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3a4005200 0x7ff3a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bad9d700 1 -- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4004850 msgr2=0x7ff3a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4004850 0x7ff3a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bad9d700 1 -- 172.21.15.144:0/1712173059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40035b0 con 0x7ff3a4003b90 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bb59e700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3a4005200 0x7ff3a400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bb59e700 1 -- 172.21.15.144:0/1712173059 reap_dead start 2023-11-28T11:53:35.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a4003b90 0x7ff3a4003f80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac001970 tx=0x7ff3a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cb550a8f4b4b84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac001460 con 0x7ff3a4003b90 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac0052f0 con 0x7ff3a4003b90 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3b9d9b700 1 -- 172.21.15.144:0/1712173059 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac001da0 con 0x7ff3a4003b90 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1712173059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a4003b90 msgr2=0x7ff3a4003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a4003b90 0x7ff3a4003f80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac001970 tx=0x7ff3a4005880 comp rx=0 tx=0).stop 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1712173059 shutdown_connections 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1712173059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a4003b90 0x7ff3a4003f80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1712173059 >> 172.21.15.144:0/1712173059 conn(0x7ff3bc06aed0 msgr2=0x7ff3bc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1712173059 shutdown_connections 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.941+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1712173059 wait complete. 2023-11-28T11:53:35.947 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.943+0000 7ff3c10a8700 1 Processor -- start 2023-11-28T11:53:35.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.943+0000 7ff3c10a8700 1 -- start start 2023-11-28T11:53:35.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4003b90 0x7ff3bc0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3bc0c6bb0 0x7ff3bc159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 0x7ff3bc142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc143330 con 0x7ff3bc142980 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0c7120 con 0x7ff3a4003b90 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0b54c0 con 0x7ff3bc0c6bb0 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 0x7ff3bc142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 0x7ff3bc142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37898/0 (socket says 172.21.15.144:37898) 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 -- 172.21.15.144:0/1262059161 learned_addr learned my addr 172.21.15.144:0/1262059161 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3ba59c700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3bc0c6bb0 0x7ff3bc159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4003b90 0x7ff3bc0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 -- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3bc0c6bb0 msgr2=0x7ff3bc159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3bc0c6bb0 0x7ff3bc159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 -- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4003b90 msgr2=0x7ff3bc0b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4003b90 0x7ff3bc0b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 -- 172.21.15.144:0/1262059161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0b35f0 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4003b90 0x7ff3bc0b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3ba59c700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3bc0c6bb0 0x7ff3bc159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3bb59e700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 0x7ff3bc142df0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0b3a70 tx=0x7ff3b4007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4017070 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400ed20 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc0b3310 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400eeb0 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.944+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc140150 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.945+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc0b5db0 con 0x7ff3bc142980 2023-11-28T11:53:35.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.946+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff3b4013420 con 0x7ff3bc142980 2023-11-28T11:53:35.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.946+0000 7ff3a37fe700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3a80682f0 0x7ff3a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:35.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.946+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff3b4085b10 con 0x7ff3bc142980 2023-11-28T11:53:35.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.946+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3a80682f0 0x7ff3a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:35.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.946+0000 7ff3bad9d700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3a80682f0 0x7ff3a806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac015040 tx=0x7ff3ac00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:35.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:35.950+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3b40522d0 con 0x7ff3bc142980 2023-11-28T11:53:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.133+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3bc068690 con 0x7ff3bc142980 2023-11-28T11:53:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.134+0000 7ff3a37fe700 1 -- 172.21.15.144:0/1262059161 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3b40558f0 con 0x7ff3bc142980 2023-11-28T11:53:36.135 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:53:36.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3a80682f0 msgr2=0x7ff3a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3a80682f0 0x7ff3a806a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac015040 tx=0x7ff3ac00a000 comp rx=0 tx=0).stop 2023-11-28T11:53:36.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 msgr2=0x7ff3bc142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 0x7ff3bc142df0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0b3a70 tx=0x7ff3b4007c90 comp rx=0 tx=0).stop 2023-11-28T11:53:36.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 shutdown_connections 2023-11-28T11:53:36.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3bc142980 0x7ff3bc142df0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3a80682f0 0x7ff3a806a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3a4003b90 0x7ff3bc0b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 --2- 172.21.15.144:0/1262059161 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3bc0c6bb0 0x7ff3bc159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 >> 172.21.15.144:0/1262059161 conn(0x7ff3bc06aed0 msgr2=0x7ff3bc13ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 shutdown_connections 2023-11-28T11:53:36.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.136+0000 7ff3c10a8700 1 -- 172.21.15.144:0/1262059161 wait complete. 2023-11-28T11:53:36.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:53:36.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.269+0000 7f1f58674700 1 Processor -- start 2023-11-28T11:53:36.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.269+0000 7f1f58674700 1 -- start start 2023-11-28T11:53:36.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f58674700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f5015cef0 0x7f1f500af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f58674700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cbdf0 0x7f1f5006b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f58674700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f500cc5f0 con 0x7f1f5006b8c0 2023-11-28T11:53:36.272 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f58674700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5006bd40 con 0x7f1f5015cef0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f58674700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5015d470 con 0x7f1f500cbdf0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f56410700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f5015cef0 0x7f1f500af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f56c11700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1f5006b8c0 0x7f1f50071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54618/0 (socket says 172.21.15.144:54618) 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f56c11700 1 -- 172.21.15.144:0/2099973966 learned_addr learned my addr 172.21.15.144:0/2099973966 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.270+0000 7f1f55c0f700 1 --2- 172.21.15.144:0/2099973966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cbdf0 0x7f1f5006b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f56410700 1 -- 172.21.15.144:0/2099973966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cbdf0 msgr2=0x7f1f5006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f56410700 1 --2- 172.21.15.144:0/2099973966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cbdf0 0x7f1f5006b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f56410700 1 -- 172.21.15.144:0/2099973966 >> v1:172.21.15.144:6789/0 conn(0x7f1f5006b8c0 legacy=0x7f1f50071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f56410700 1 -- 172.21.15.144:0/2099973966 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5015d750 con 0x7f1f5015cef0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f5540e700 1 -- 172.21.15.144:0/2099973966 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321995997 0 0) 0x7f1f500cc5f0 con 0x7f1f5006b8c0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f56410700 1 --2- 172.21.15.144:0/2099973966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015cef0 0x7f1f500af400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009840 tx=0x7f1f4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b29166a408355c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:36.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f5540e700 1 -- 172.21.15.144:0/2099973966 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c011830 con 0x7f1f5015cef0 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f5540e700 1 -- 172.21.15.144:0/2099973966 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c011990 con 0x7f1f5015cef0 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f5540e700 1 -- 172.21.15.144:0/2099973966 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c011d20 con 0x7f1f5015cef0 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 -- 172.21.15.144:0/2099973966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015cef0 msgr2=0x7f1f500af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 --2- 172.21.15.144:0/2099973966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015cef0 0x7f1f500af400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009840 tx=0x7f1f4c009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 -- 172.21.15.144:0/2099973966 shutdown_connections 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 --2- 172.21.15.144:0/2099973966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015cef0 0x7f1f500af400 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 --2- 172.21.15.144:0/2099973966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f500cbdf0 0x7f1f5006b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 -- 172.21.15.144:0/2099973966 >> 172.21.15.144:0/2099973966 conn(0x7f1f500d0000 msgr2=0x7f1f500d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 -- 172.21.15.144:0/2099973966 shutdown_connections 2023-11-28T11:53:36.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.271+0000 7f1f58674700 1 -- 172.21.15.144:0/2099973966 wait complete. 2023-11-28T11:53:36.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 Processor -- start 2023-11-28T11:53:36.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 -- start start 2023-11-28T11:53:36.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 0x7f1f5006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f5015cef0 0x7f1f5006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f50155fd0 con 0x7f1f500cbdf0 2023-11-28T11:53:36.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f500b5bb0 con 0x7f1f5006b8c0 2023-11-28T11:53:36.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f58674700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f501f5e90 con 0x7f1f5015cef0 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.273+0000 7f1f55c0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 0x7f1f5006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 0x7f1f5006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37910/0 (socket says 172.21.15.144:37910) 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 -- 172.21.15.144:0/321582438 learned_addr learned my addr 172.21.15.144:0/321582438 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f56410700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f56c11700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f5015cef0 0x7f1f5006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 -- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f5015cef0 msgr2=0x7f1f5006a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f5015cef0 0x7f1f5006a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 -- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 msgr2=0x7f1f500b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 -- 172.21.15.144:0/321582438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0094a0 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f55c0f700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 0x7f1f5006a270 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000e9b0 tx=0x7f1f4000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40009710 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4000c850 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5015dd10 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4000cbc0 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.274+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5015da30 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.275+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5015dff0 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.276+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f40019030 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.276+0000 7f1f46ffd700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f340682f0 0x7f1f3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.276+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1f40085cf0 con 0x7f1f500cbdf0 2023-11-28T11:53:36.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.276+0000 7f1f56410700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f340682f0 0x7f1f3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.276+0000 7f1f56410700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f340682f0 0x7f1f3406a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ca970 tx=0x7f1f4c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:36.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.280+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f40052510 con 0x7f1f500cbdf0 2023-11-28T11:53:36.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.480+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1f501f6d80 con 0x7f1f500cbdf0 2023-11-28T11:53:36.481 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.480+0000 7f1f46ffd700 1 -- 172.21.15.144:0/321582438 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1f40021080 con 0x7f1f500cbdf0 2023-11-28T11:53:36.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f340682f0 msgr2=0x7f1f3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f340682f0 0x7f1f3406a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1f500ca970 tx=0x7f1f4c008030 comp rx=0 tx=0).stop 2023-11-28T11:53:36.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 msgr2=0x7f1f5006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 0x7f1f5006a270 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000e9b0 tx=0x7f1f4000ed70 comp rx=0 tx=0).stop 2023-11-28T11:53:36.483 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 shutdown_connections 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1f340682f0 0x7f1f3406a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1f5015cef0 0x7f1f5006a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f5006b8c0 0x7f1f500b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 --2- 172.21.15.144:0/321582438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f500cbdf0 0x7f1f5006a270 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 >> 172.21.15.144:0/321582438 conn(0x7f1f500d0000 msgr2=0x7f1f500b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 shutdown_connections 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.482+0000 7f1f58674700 1 -- 172.21.15.144:0/321582438 wait complete. 2023-11-28T11:53:36.484 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:53:36.495 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:53:36.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:36.495 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:36.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:53:36.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:36.555 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:53:36.584 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:53:36.584 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:36.604 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:53:36.604 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:53:36.604 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:36.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:53:36.642 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:53:36.642 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:36.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:53:36.662 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:36.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.803+0000 7fed1edb1700 1 Processor -- start 2023-11-28T11:53:36.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.803+0000 7fed1edb1700 1 -- start start 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1edb1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180ce040 0x7fed1815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1edb1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fed180d0a00 0x7fed1806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1edb1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 0x7fed180b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1edb1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180b32f0 con 0x7fed180b2930 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1edb1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1806b420 con 0x7fed180d0a00 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1edb1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806b700 con 0x7fed180ce040 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1d34e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 0x7fed180b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1cb4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180ce040 0x7fed1815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1d34e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 0x7fed180b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37920/0 (socket says 172.21.15.144:37920) 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed1d34e700 1 -- 172.21.15.144:0/3848170417 learned_addr learned my addr 172.21.15.144:0/3848170417 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.804+0000 7fed17fff700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.177:3300/0 conn(0x7fed180d0a00 0x7fed1806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1cb4d700 1 -- 172.21.15.144:0/3848170417 >> v2:172.21.15.177:3300/0 conn(0x7fed180d0a00 msgr2=0x7fed1806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1cb4d700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.177:3300/0 conn(0x7fed180d0a00 0x7fed1806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1cb4d700 1 -- 172.21.15.144:0/3848170417 >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 msgr2=0x7fed180b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1cb4d700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 0x7fed180b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1cb4d700 1 -- 172.21.15.144:0/3848170417 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1806b9e0 con 0x7fed180ce040 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed17fff700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.177:3300/0 conn(0x7fed180d0a00 0x7fed1806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1d34e700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 0x7fed180b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:36.807 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.805+0000 7fed1cb4d700 1 --2- 172.21.15.144:0/3848170417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180ce040 0x7fed1815df10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed18067c60 tx=0x7fed0c009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96c082c31691fb8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:36.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.806+0000 7fed177fe700 1 -- 172.21.15.144:0/3848170417 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c005e30 con 0x7fed180ce040 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.806+0000 7fed177fe700 1 -- 172.21.15.144:0/3848170417 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c006030 con 0x7fed180ce040 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.806+0000 7fed177fe700 1 -- 172.21.15.144:0/3848170417 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0063c0 con 0x7fed180ce040 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 -- 172.21.15.144:0/3848170417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180ce040 msgr2=0x7fed1815df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 --2- 172.21.15.144:0/3848170417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180ce040 0x7fed1815df10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed18067c60 tx=0x7fed0c009b00 comp rx=0 tx=0).stop 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 -- 172.21.15.144:0/3848170417 shutdown_connections 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.144:3300/0 conn(0x7fed180b2930 0x7fed180b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 --2- 172.21.15.144:0/3848170417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180ce040 0x7fed1815df10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 --2- 172.21.15.144:0/3848170417 >> v2:172.21.15.177:3300/0 conn(0x7fed180d0a00 0x7fed1806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 -- 172.21.15.144:0/3848170417 >> 172.21.15.144:0/3848170417 conn(0x7fed1806aed0 msgr2=0x7fed1806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 -- 172.21.15.144:0/3848170417 shutdown_connections 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.807+0000 7fed1edb1700 1 -- 172.21.15.144:0/3848170417 wait complete. 2023-11-28T11:53:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.808+0000 7fed1edb1700 1 Processor -- start 2023-11-28T11:53:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.808+0000 7fed1edb1700 1 -- start start 2023-11-28T11:53:36.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1edb1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180b2930 0x7fed180c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1edb1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed180ce040 0x7fed181f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1edb1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 0x7fed180cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1edb1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed181f6380 con 0x7fed180d0a00 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1edb1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c61d0 con 0x7fed180ce040 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1edb1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed181f5700 con 0x7fed180b2930 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 0x7fed180cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 0x7fed180cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37926/0 (socket says 172.21.15.144:37926) 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 -- 172.21.15.144:0/1618854344 learned_addr learned my addr 172.21.15.144:0/1618854344 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1cb4d700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180b2930 0x7fed180c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed17fff700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed180ce040 0x7fed181f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 -- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180b2930 msgr2=0x7fed180c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.813 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180b2930 0x7fed180c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 -- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed180ce040 msgr2=0x7fed181f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed180ce040 0x7fed181f1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.809+0000 7fed1d34e700 1 -- 172.21.15.144:0/1618854344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed10009b30 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.810+0000 7fed1d34e700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 0x7fed180cc530 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fed1000efc0 tx=0x7fed1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.810+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10018070 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.810+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1000f040 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.810+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed040094a0 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.810+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10003c40 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.810+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c009430 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.811+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1806bfa0 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.811+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed1000c880 con 0x7fed180d0a00 2023-11-28T11:53:36.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.811+0000 7fed157fa700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fecfc0682f0 0x7fecfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.811+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fed10085f50 con 0x7fed180d0a00 2023-11-28T11:53:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.811+0000 7fed1cb4d700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fecfc0682f0 0x7fecfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.812+0000 7fed1cb4d700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fecfc0682f0 0x7fecfc06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed0c000c00 tx=0x7fed0c008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:36.816+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fed100526f0 con 0x7fed180d0a00 2023-11-28T11:53:37.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.043+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed1806fd90 con 0x7fed180d0a00 2023-11-28T11:53:37.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.043+0000 7fed157fa700 1 -- 172.21.15.144:0/1618854344 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fed10009ce0 con 0x7fed180d0a00 2023-11-28T11:53:37.044 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:37.044 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:53:37.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fecfc0682f0 msgr2=0x7fecfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fecfc0682f0 0x7fecfc06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed0c000c00 tx=0x7fed0c008070 comp rx=0 tx=0).stop 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 msgr2=0x7fed180cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 0x7fed180cc530 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fed1000efc0 tx=0x7fed1000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 shutdown_connections 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed180d0a00 0x7fed180cc530 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fecfc0682f0 0x7fecfc06a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fed180b2930 0x7fed180c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 --2- 172.21.15.144:0/1618854344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed180ce040 0x7fed181f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.045+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 >> 172.21.15.144:0/1618854344 conn(0x7fed1806aed0 msgr2=0x7fed18070030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.046+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 shutdown_connections 2023-11-28T11:53:37.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.046+0000 7fed1edb1700 1 -- 172.21.15.144:0/1618854344 wait complete. 2023-11-28T11:53:37.048 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:37.058 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:37.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.177+0000 7f069c862700 1 Processor -- start 2023-11-28T11:53:37.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.178+0000 7f069c862700 1 -- start start 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.178+0000 7f069c862700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069815cef0 0x7f06980af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.178+0000 7f069c862700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f06980cbdf0 0x7f069806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.178+0000 7f069c862700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980cc5f0 con 0x7f069806b8c0 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069c862700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069806bd40 con 0x7f06980cbdf0 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069c862700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069815d470 con 0x7f069815cef0 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f0696d9d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f069806b8c0 0x7f0698071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54644/0 (socket says 172.21.15.144:54644) 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f0696d9d700 1 -- 172.21.15.144:0/832972510 learned_addr learned my addr 172.21.15.144:0/832972510 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069659c700 1 --2- 172.21.15.144:0/832972510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069815cef0 0x7f06980af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f0695d9b700 1 --2- 172.21.15.144:0/832972510 >> v2:172.21.15.177:3300/0 conn(0x7f06980cbdf0 0x7f069806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069559a700 1 -- 172.21.15.144:0/832972510 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577784268 0 0) 0x7f06980cc5f0 con 0x7f069806b8c0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069559a700 1 -- 172.21.15.144:0/832972510 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06800035b0 con 0x7f069806b8c0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069659c700 1 -- 172.21.15.144:0/832972510 >> v2:172.21.15.177:3300/0 conn(0x7f06980cbdf0 msgr2=0x7f069806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069659c700 1 --2- 172.21.15.144:0/832972510 >> v2:172.21.15.177:3300/0 conn(0x7f06980cbdf0 0x7f069806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069659c700 1 -- 172.21.15.144:0/832972510 >> v1:172.21.15.144:6789/0 conn(0x7f069806b8c0 legacy=0x7f0698071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069659c700 1 -- 172.21.15.144:0/832972510 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069815d750 con 0x7f069815cef0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f0695d9b700 1 --2- 172.21.15.144:0/832972510 >> v2:172.21.15.177:3300/0 conn(0x7f06980cbdf0 0x7f069806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069659c700 1 --2- 172.21.15.144:0/832972510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069815cef0 0x7f06980af400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0688009840 tx=0x7f0688009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2556f8214c1d4623 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069559a700 1 -- 172.21.15.144:0/832972510 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688011830 con 0x7f069815cef0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069559a700 1 -- 172.21.15.144:0/832972510 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688011990 con 0x7f069815cef0 2023-11-28T11:53:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.179+0000 7f069559a700 1 -- 172.21.15.144:0/832972510 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688011d20 con 0x7f069815cef0 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 -- 172.21.15.144:0/832972510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069815cef0 msgr2=0x7f06980af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 --2- 172.21.15.144:0/832972510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069815cef0 0x7f06980af400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0688009840 tx=0x7f0688009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 -- 172.21.15.144:0/832972510 shutdown_connections 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 --2- 172.21.15.144:0/832972510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069815cef0 0x7f06980af400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 --2- 172.21.15.144:0/832972510 >> v2:172.21.15.177:3300/0 conn(0x7f06980cbdf0 0x7f069806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 -- 172.21.15.144:0/832972510 >> 172.21.15.144:0/832972510 conn(0x7f06980d0000 msgr2=0x7f06980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 -- 172.21.15.144:0/832972510 shutdown_connections 2023-11-28T11:53:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.180+0000 7f069c862700 1 -- 172.21.15.144:0/832972510 wait complete. 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 Processor -- start 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 -- start start 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069806b8c0 0x7f0698158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 0x7f0698154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069815cef0 0x7f06980b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06980b2e30 con 0x7f06980cbdf0 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698159480 con 0x7f069815cef0 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f069c862700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698159760 con 0x7f069806b8c0 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f0695d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 0x7f0698154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f0695d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 0x7f0698154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37936/0 (socket says 172.21.15.144:37936) 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.182+0000 7f0695d9b700 1 -- 172.21.15.144:0/1044744350 learned_addr learned my addr 172.21.15.144:0/1044744350 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f069659c700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069806b8c0 0x7f0698158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0696d9d700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069815cef0 0x7f06980b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0695d9b700 1 -- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069806b8c0 msgr2=0x7f0698158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0695d9b700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069806b8c0 0x7f0698158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0695d9b700 1 -- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069815cef0 msgr2=0x7f06980b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0695d9b700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069815cef0 0x7f06980b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0695d9b700 1 -- 172.21.15.144:0/1044744350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0094a0 con 0x7f06980cbdf0 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0696d9d700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069815cef0 0x7f06980b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0695d9b700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 0x7f0698154c00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f068c00ebe0 tx=0x7f068c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c009800 con 0x7f06980cbdf0 2023-11-28T11:53:37.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c00c910 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06880094a0 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00cc80 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.183+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069815dd10 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.184+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069815da30 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.184+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f068c019030 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.185+0000 7f0686ffd700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06780682f0 0x7f067806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.185+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f068c085ca0 con 0x7f06980cbdf0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.185+0000 7f069659c700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06780682f0 0x7f067806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.185+0000 7f069659c700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06780682f0 0x7f067806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0688000c00 tx=0x7f0688008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.189+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f068c08a070 con 0x7f06980cbdf0 2023-11-28T11:53:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.372+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06980cf170 con 0x7f06980cbdf0 2023-11-28T11:53:37.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.373+0000 7f0686ffd700 1 -- 172.21.15.144:0/1044744350 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18009 (secure 0 0 0) 0x7f068c017090 con 0x7f06980cbdf0 2023-11-28T11:53:37.374 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:37.374 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":48,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:05.732387+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:37.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06780682f0 msgr2=0x7f067806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06780682f0 0x7f067806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0688000c00 tx=0x7f0688008030 comp rx=0 tx=0).stop 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 msgr2=0x7f0698154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 0x7f0698154c00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f068c00ebe0 tx=0x7f068c00eef0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 shutdown_connections 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f06780682f0 0x7f067806a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f069815cef0 0x7f06980b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f069806b8c0 0x7f0698158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 --2- 172.21.15.144:0/1044744350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06980cbdf0 0x7f0698154c00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 >> 172.21.15.144:0/1044744350 conn(0x7f06980d0000 msgr2=0x7f06980afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:37.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 shutdown_connections 2023-11-28T11:53:37.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.375+0000 7f069c862700 1 -- 172.21.15.144:0/1044744350 wait complete. 2023-11-28T11:53:37.388 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:52:48.228399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:52:49.180034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:53:37.389 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:53:37.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.510+0000 7f42439ae700 1 Processor -- start 2023-11-28T11:53:37.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.510+0000 7f42439ae700 1 -- start start 2023-11-28T11:53:37.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f42439ae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f423c071040 0x7f423c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f42439ae700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f423c160540 0x7f423c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f42439ae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c071950 con 0x7f423c06dee0 2023-11-28T11:53:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f42439ae700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c15d060 con 0x7f423c160540 2023-11-28T11:53:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f42439ae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c15d340 con 0x7f423c071040 2023-11-28T11:53:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f423c160540 0x7f423c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f423c160540 0x7f423c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37946/0 (socket says 172.21.15.144:37946) 2023-11-28T11:53:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 -- 172.21.15.144:0/3954747075 learned_addr learned my addr 172.21.15.144:0/3954747075 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4240f49700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f423c06dee0 0x7f423c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54656/0 (socket says 172.21.15.144:54656) 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f424174a700 1 --2- 172.21.15.144:0/3954747075 >> v2:172.21.15.177:3300/0 conn(0x7f423c071040 0x7f423c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 -- 172.21.15.144:0/3954747075 >> v2:172.21.15.177:3300/0 conn(0x7f423c071040 msgr2=0x7f423c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 --2- 172.21.15.144:0/3954747075 >> v2:172.21.15.177:3300/0 conn(0x7f423c071040 0x7f423c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 -- 172.21.15.144:0/3954747075 >> v1:172.21.15.144:6789/0 conn(0x7f423c06dee0 legacy=0x7f423c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4233fff700 1 -- 172.21.15.144:0/3954747075 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443039452 0 0) 0x7f423c071950 con 0x7f423c06dee0 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 -- 172.21.15.144:0/3954747075 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c15d620 con 0x7f423c160540 2023-11-28T11:53:37.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.511+0000 7f4241f4b700 1 --2- 172.21.15.144:0/3954747075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c160540 0x7f423c160980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f423c0d0f50 tx=0x7f423400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47250e048715abe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f4233fff700 1 -- 172.21.15.144:0/3954747075 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400e070 con 0x7f423c160540 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f4233fff700 1 -- 172.21.15.144:0/3954747075 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400b9c0 con 0x7f423c160540 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f4233fff700 1 -- 172.21.15.144:0/3954747075 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400bd70 con 0x7f423c160540 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 -- 172.21.15.144:0/3954747075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c160540 msgr2=0x7f423c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 --2- 172.21.15.144:0/3954747075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c160540 0x7f423c160980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f423c0d0f50 tx=0x7f423400b330 comp rx=0 tx=0).stop 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 -- 172.21.15.144:0/3954747075 shutdown_connections 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 --2- 172.21.15.144:0/3954747075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c160540 0x7f423c160980 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 --2- 172.21.15.144:0/3954747075 >> v2:172.21.15.177:3300/0 conn(0x7f423c071040 0x7f423c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 -- 172.21.15.144:0/3954747075 >> 172.21.15.144:0/3954747075 conn(0x7f423c0d0000 msgr2=0x7f423c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:37.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 -- 172.21.15.144:0/3954747075 shutdown_connections 2023-11-28T11:53:37.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.512+0000 7f42439ae700 1 -- 172.21.15.144:0/3954747075 wait complete. 2023-11-28T11:53:37.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.513+0000 7f42439ae700 1 Processor -- start 2023-11-28T11:53:37.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.513+0000 7f42439ae700 1 -- start start 2023-11-28T11:53:37.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f42439ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f423c06dee0 0x7f423c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f42439ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 0x7f423c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f42439ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423c160540 0x7f423c0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f42439ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c14ea10 con 0x7f423c071040 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f42439ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0c48f0 con 0x7f423c06dee0 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f42439ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0c4bd0 con 0x7f423c160540 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4240f49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 0x7f423c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4240f49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 0x7f423c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37950/0 (socket says 172.21.15.144:37950) 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4240f49700 1 -- 172.21.15.144:0/898507047 learned_addr learned my addr 172.21.15.144:0/898507047 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:37.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f424174a700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f423c06dee0 0x7f423c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4241f4b700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423c160540 0x7f423c0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4240f49700 1 -- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423c160540 msgr2=0x7f423c0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4240f49700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423c160540 0x7f423c0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.514+0000 7f4240f49700 1 -- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f423c06dee0 msgr2=0x7f423c14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f4240f49700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f423c06dee0 0x7f423c14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f4240f49700 1 -- 172.21.15.144:0/898507047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234009b30 con 0x7f423c071040 2023-11-28T11:53:37.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f424174a700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f423c06dee0 0x7f423c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f4240f49700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 0x7f423c0b2e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f423800c550 tx=0x7f423800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238004c80 con 0x7f423c071040 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238004e00 con 0x7f423c071040 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c15ccd0 con 0x7f423c071040 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238007310 con 0x7f423c071040 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.515+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c15d900 con 0x7f423c071040 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.516+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c06a270 con 0x7f423c071040 2023-11-28T11:53:37.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.516+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f423801a020 con 0x7f423c071040 2023-11-28T11:53:37.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.517+0000 7f4231ffb700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f42200682f0 0x7f422006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.517+0000 7f424174a700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f42200682f0 0x7f422006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.517+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4238085880 con 0x7f423c071040 2023-11-28T11:53:37.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.517+0000 7f424174a700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f42200682f0 0x7f422006a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f422c009820 tx=0x7f422c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.521+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42380520a0 con 0x7f423c071040 2023-11-28T11:53:37.700 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.699+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f423c0720a0 con 0x7f423c071040 2023-11-28T11:53:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.700+0000 7f4231ffb700 1 -- 172.21.15.144:0/898507047 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f42380556c0 con 0x7f423c071040 2023-11-28T11:53:37.701 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:53:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f42200682f0 msgr2=0x7f422006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f42200682f0 0x7f422006a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f422c009820 tx=0x7f422c008040 comp rx=0 tx=0).stop 2023-11-28T11:53:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 msgr2=0x7f423c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 0x7f423c0b2e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f423800c550 tx=0x7f423800c910 comp rx=0 tx=0).stop 2023-11-28T11:53:37.703 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 shutdown_connections 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f42200682f0 0x7f422006a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f423c160540 0x7f423c0b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f423c06dee0 0x7f423c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 --2- 172.21.15.144:0/898507047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f423c071040 0x7f423c0b2e30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 >> 172.21.15.144:0/898507047 conn(0x7f423c0d0000 msgr2=0x7f423c0732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 shutdown_connections 2023-11-28T11:53:37.704 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.702+0000 7f42439ae700 1 -- 172.21.15.144:0/898507047 wait complete. 2023-11-28T11:53:37.715 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:53:37.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.840+0000 7f0d8b70b700 1 Processor -- start 2023-11-28T11:53:37.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.840+0000 7f0d8b70b700 1 -- start start 2023-11-28T11:53:37.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d8b70b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0d840ce040 0x7f0d8415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d8b70b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840d0a00 0x7f0d8406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d8b70b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8406b420 con 0x7f0d840b2930 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d8b70b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8406b700 con 0x7f0d840ce040 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d8b70b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8406b9e0 con 0x7f0d840d0a00 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d88ca6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840d0a00 0x7f0d8406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d894a7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0d840ce040 0x7f0d8415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d89ca8700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0d840b2930 0x7f0d8406c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55162/0 (socket says 172.21.15.144:55162) 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.841+0000 7f0d89ca8700 1 -- 172.21.15.144:0/2833112575 learned_addr learned my addr 172.21.15.144:0/2833112575 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d88ca6700 1 -- 172.21.15.144:0/2833112575 >> v2:172.21.15.177:3300/0 conn(0x7f0d840ce040 msgr2=0x7f0d8415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d88ca6700 1 --2- 172.21.15.144:0/2833112575 >> v2:172.21.15.177:3300/0 conn(0x7f0d840ce040 0x7f0d8415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d88ca6700 1 -- 172.21.15.144:0/2833112575 >> v1:172.21.15.177:6789/0 conn(0x7f0d840b2930 legacy=0x7f0d8406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d88ca6700 1 -- 172.21.15.144:0/2833112575 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8406bcc0 con 0x7f0d840d0a00 2023-11-28T11:53:37.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d894a7700 1 --2- 172.21.15.144:0/2833112575 >> v2:172.21.15.177:3300/0 conn(0x7f0d840ce040 0x7f0d8415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:37.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d88ca6700 1 --2- 172.21.15.144:0/2833112575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840d0a00 0x7f0d8406f850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009840 tx=0x7f0d74009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb33c5e4fb3f2697 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d7bfff700 1 -- 172.21.15.144:0/2833112575 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74011830 con 0x7f0d840d0a00 2023-11-28T11:53:37.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d7bfff700 1 -- 172.21.15.144:0/2833112575 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d74011990 con 0x7f0d840d0a00 2023-11-28T11:53:37.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.842+0000 7f0d7bfff700 1 -- 172.21.15.144:0/2833112575 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74011d20 con 0x7f0d840d0a00 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 -- 172.21.15.144:0/2833112575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840d0a00 msgr2=0x7f0d8406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/2833112575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840d0a00 0x7f0d8406f850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009840 tx=0x7f0d74009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 -- 172.21.15.144:0/2833112575 shutdown_connections 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/2833112575 >> v2:172.21.15.177:3300/0 conn(0x7f0d840ce040 0x7f0d8415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/2833112575 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840d0a00 0x7f0d8406f850 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 -- 172.21.15.144:0/2833112575 >> 172.21.15.144:0/2833112575 conn(0x7f0d8406aed0 msgr2=0x7f0d8406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 -- 172.21.15.144:0/2833112575 shutdown_connections 2023-11-28T11:53:37.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.843+0000 7f0d8b70b700 1 -- 172.21.15.144:0/2833112575 wait complete. 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.844+0000 7f0d8b70b700 1 Processor -- start 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.844+0000 7f0d8b70b700 1 -- start start 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840b2930 0x7f0d840c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840ce040 0x7f0d8414fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 0x7f0d840b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841539c0 con 0x7f0d840d0a00 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84153ca0 con 0x7f0d840ce040 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84153f80 con 0x7f0d840b2930 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 0x7f0d840b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 0x7f0d840b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37966/0 (socket says 172.21.15.144:37966) 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 -- 172.21.15.144:0/54232450 learned_addr learned my addr 172.21.15.144:0/54232450 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:37.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d894a7700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840b2930 0x7f0d840c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d88ca6700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840ce040 0x7f0d8414fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 -- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840b2930 msgr2=0x7f0d840c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840b2930 0x7f0d840c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 -- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840ce040 msgr2=0x7f0d8414fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840ce040 0x7f0d8414fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 -- 172.21.15.144:0/54232450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009430 con 0x7f0d840d0a00 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d88ca6700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840ce040 0x7f0d8414fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d89ca8700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 0x7f0d840b2db0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00ca80 tx=0x7f0d7c00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00a9e0 con 0x7f0d840d0a00 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00ab40 con 0x7f0d840d0a00 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d740094a0 con 0x7f0d840d0a00 2023-11-28T11:53:37.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.845+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c012400 con 0x7f0d840d0a00 2023-11-28T11:53:37.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.846+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8406fd90 con 0x7f0d840d0a00 2023-11-28T11:53:37.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.847+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8406bfa0 con 0x7f0d840d0a00 2023-11-28T11:53:37.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.847+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d7c019030 con 0x7f0d840d0a00 2023-11-28T11:53:37.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.847+0000 7f0d79ffb700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d680682f0 0x7f0d6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:37.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.847+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0d7c085640 con 0x7f0d840d0a00 2023-11-28T11:53:37.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.847+0000 7f0d894a7700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d680682f0 0x7f0d6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:37.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.848+0000 7f0d894a7700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d680682f0 0x7f0d6806a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009cb0 tx=0x7f0d800091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:37.852 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:37.851+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d7c04b5e0 con 0x7f0d840d0a00 2023-11-28T11:53:38.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.034+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d8406ff20 con 0x7f0d840d0a00 2023-11-28T11:53:38.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.034+0000 7f0d79ffb700 1 -- 172.21.15.144:0/54232450 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d7c051f10 con 0x7f0d840d0a00 2023-11-28T11:53:38.035 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:53:38.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d680682f0 msgr2=0x7f0d6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d680682f0 0x7f0d6806a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009cb0 tx=0x7f0d800091c0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 msgr2=0x7f0d840b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 0x7f0d840b2db0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00ca80 tx=0x7f0d7c00cd90 comp rx=0 tx=0).stop 2023-11-28T11:53:38.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 shutdown_connections 2023-11-28T11:53:38.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d840d0a00 0x7f0d840b2db0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0d680682f0 0x7f0d6806a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0d840b2930 0x7f0d840c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 --2- 172.21.15.144:0/54232450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d840ce040 0x7f0d8414fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 >> 172.21.15.144:0/54232450 conn(0x7f0d8406aed0 msgr2=0x7f0d84151440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 shutdown_connections 2023-11-28T11:53:38.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.036+0000 7f0d8b70b700 1 -- 172.21.15.144:0/54232450 wait complete. 2023-11-28T11:53:38.049 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:53:38.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.171+0000 7f59cdbee700 1 Processor -- start 2023-11-28T11:53:38.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.171+0000 7f59cdbee700 1 -- start start 2023-11-28T11:53:38.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.171+0000 7f59cdbee700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59c8071040 0x7f59c8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59cdbee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f59c806dee0 0x7f59c806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59cdbee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c806cdc0 0x7f59c806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59cdbee700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c8071950 con 0x7f59c8071040 2023-11-28T11:53:38.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59cdbee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c815d060 con 0x7f59c806dee0 2023-11-28T11:53:38.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59cdbee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c815d340 con 0x7f59c806cdc0 2023-11-28T11:53:38.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59c8071040 0x7f59c8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59c8071040 0x7f59c8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37970/0 (socket says 172.21.15.144:37970) 2023-11-28T11:53:38.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 -- 172.21.15.144:0/743668487 learned_addr learned my addr 172.21.15.144:0/743668487 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:38.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c7fff700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c806cdc0 0x7f59c806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 -- 172.21.15.144:0/743668487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c806cdc0 msgr2=0x7f59c806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c806cdc0 0x7f59c806d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 -- 172.21.15.144:0/743668487 >> v2:172.21.15.177:3300/0 conn(0x7f59c806dee0 msgr2=0x7f59c806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c6ffd700 1 --2- 172.21.15.144:0/743668487 >> v2:172.21.15.177:3300/0 conn(0x7f59c806dee0 0x7f59c806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 --2- 172.21.15.144:0/743668487 >> v2:172.21.15.177:3300/0 conn(0x7f59c806dee0 0x7f59c806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 -- 172.21.15.144:0/743668487 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c815d620 con 0x7f59c8071040 2023-11-28T11:53:38.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c7fff700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c806cdc0 0x7f59c806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:38.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c6ffd700 1 --2- 172.21.15.144:0/743668487 >> v2:172.21.15.177:3300/0 conn(0x7f59c806dee0 0x7f59c806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c77fe700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c8071040 0x7f59c8071410 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f59c806fb00 tx=0x7f59b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78be267db4ab5e85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c67fc700 1 -- 172.21.15.144:0/743668487 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800b870 con 0x7f59c8071040 2023-11-28T11:53:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c67fc700 1 -- 172.21.15.144:0/743668487 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b800b9d0 con 0x7f59c8071040 2023-11-28T11:53:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.172+0000 7f59c67fc700 1 -- 172.21.15.144:0/743668487 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800bd60 con 0x7f59c8071040 2023-11-28T11:53:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 -- 172.21.15.144:0/743668487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c8071040 msgr2=0x7f59c8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c8071040 0x7f59c8071410 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f59c806fb00 tx=0x7f59b8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 -- 172.21.15.144:0/743668487 shutdown_connections 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c806cdc0 0x7f59c806d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 --2- 172.21.15.144:0/743668487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c8071040 0x7f59c8071410 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 --2- 172.21.15.144:0/743668487 >> v2:172.21.15.177:3300/0 conn(0x7f59c806dee0 0x7f59c806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 -- 172.21.15.144:0/743668487 >> 172.21.15.144:0/743668487 conn(0x7f59c80d0000 msgr2=0x7f59c80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 -- 172.21.15.144:0/743668487 shutdown_connections 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.173+0000 7f59cdbee700 1 -- 172.21.15.144:0/743668487 wait complete. 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.174+0000 7f59cdbee700 1 Processor -- start 2023-11-28T11:53:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.174+0000 7f59cdbee700 1 -- start start 2023-11-28T11:53:38.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59cdbee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 0x7f59c815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59cdbee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59c806dee0 0x7f59c8146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59cdbee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c8071040 0x7f59c8147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59cdbee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80b5720 con 0x7f59c806cdc0 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59cdbee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c8147960 con 0x7f59c806dee0 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59cdbee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c81f6730 con 0x7f59c8071040 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 0x7f59c815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 0x7f59c815e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37976/0 (socket says 172.21.15.144:37976) 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59c77fe700 1 -- 172.21.15.144:0/2464137354 learned_addr learned my addr 172.21.15.144:0/2464137354 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59c7fff700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c8071040 0x7f59c8147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.175+0000 7f59c6ffd700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59c806dee0 0x7f59c8146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59c77fe700 1 -- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c8071040 msgr2=0x7f59c8147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59c77fe700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c8071040 0x7f59c8147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59c77fe700 1 -- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59c806dee0 msgr2=0x7f59c8146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59c77fe700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59c806dee0 0x7f59c8146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59c77fe700 1 -- 172.21.15.144:0/2464137354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8009550 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59c77fe700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 0x7f59c815e7e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f59b8014040 tx=0x7f59b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8004500 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b80036a0 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c806e870 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8003a30 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.176+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c815d900 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.177+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c815ccd0 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.177+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f59b802c070 con 0x7f59c806cdc0 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.178+0000 7f59affff700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f59b40682f0 0x7f59b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.178+0000 7f59c6ffd700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f59b40682f0 0x7f59b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.178+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f59b8020030 con 0x7f59c806cdc0 2023-11-28T11:53:38.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.178+0000 7f59c6ffd700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f59b40682f0 0x7f59b406a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f59c8071580 tx=0x7f59bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.182+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f59b8053740 con 0x7f59c806cdc0 2023-11-28T11:53:38.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.367+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59c80720a0 con 0x7f59c806cdc0 2023-11-28T11:53:38.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.368+0000 7f59affff700 1 -- 172.21.15.144:0/2464137354 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59b802a020 con 0x7f59c806cdc0 2023-11-28T11:53:38.369 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:53:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f59b40682f0 msgr2=0x7f59b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f59b40682f0 0x7f59b406a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f59c8071580 tx=0x7f59bc008040 comp rx=0 tx=0).stop 2023-11-28T11:53:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 msgr2=0x7f59c815e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 0x7f59c815e7e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f59b8014040 tx=0x7f59b8004750 comp rx=0 tx=0).stop 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 shutdown_connections 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59c806cdc0 0x7f59c815e7e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f59b40682f0 0x7f59b406a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59c8071040 0x7f59c8147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 --2- 172.21.15.144:0/2464137354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59c806dee0 0x7f59c8146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 >> 172.21.15.144:0/2464137354 conn(0x7f59c80d0000 msgr2=0x7f59c815f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:38.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 shutdown_connections 2023-11-28T11:53:38.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.370+0000 7f59cdbee700 1 -- 172.21.15.144:0/2464137354 wait complete. 2023-11-28T11:53:38.383 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:53:38.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.504+0000 7f670fb5a700 1 Processor -- start 2023-11-28T11:53:38.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 -- start start 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6708159050 0x7f67080cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6708071fb0 0x7f670806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f670806a7b0 0x7f67080b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6708072750 con 0x7f6708159050 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670806ac10 con 0x7f6708071fb0 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.505+0000 7f670fb5a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080cc7a0 con 0x7f670806a7b0 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6708159050 0x7f67080cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6708159050 0x7f67080cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37984/0 (socket says 172.21.15.144:37984) 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 -- 172.21.15.144:0/668598521 learned_addr learned my addr 172.21.15.144:0/668598521 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670e0f7700 1 --2- 172.21.15.144:0/668598521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f670806a7b0 0x7f67080b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d0f5700 1 --2- 172.21.15.144:0/668598521 >> v2:172.21.15.177:3300/0 conn(0x7f6708071fb0 0x7f670806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 -- 172.21.15.144:0/668598521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f670806a7b0 msgr2=0x7f67080b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 --2- 172.21.15.144:0/668598521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f670806a7b0 0x7f67080b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 -- 172.21.15.144:0/668598521 >> v2:172.21.15.177:3300/0 conn(0x7f6708071fb0 msgr2=0x7f670806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 --2- 172.21.15.144:0/668598521 >> v2:172.21.15.177:3300/0 conn(0x7f6708071fb0 0x7f670806a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 -- 172.21.15.144:0/668598521 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6708159420 con 0x7f6708159050 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670d8f6700 1 --2- 172.21.15.144:0/668598521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f67080cfe10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6708066dd0 tx=0x7f66f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a3bbc2d86acb27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670c8f4700 1 -- 172.21.15.144:0/668598521 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800b870 con 0x7f6708159050 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670c8f4700 1 -- 172.21.15.144:0/668598521 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f800b9d0 con 0x7f6708159050 2023-11-28T11:53:38.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.506+0000 7f670c8f4700 1 -- 172.21.15.144:0/668598521 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800bd60 con 0x7f6708159050 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 -- 172.21.15.144:0/668598521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 msgr2=0x7f67080cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 --2- 172.21.15.144:0/668598521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f67080cfe10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6708066dd0 tx=0x7f66f8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 -- 172.21.15.144:0/668598521 shutdown_connections 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 --2- 172.21.15.144:0/668598521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f670806a7b0 0x7f67080b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 --2- 172.21.15.144:0/668598521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f67080cfe10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 --2- 172.21.15.144:0/668598521 >> v2:172.21.15.177:3300/0 conn(0x7f6708071fb0 0x7f670806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 -- 172.21.15.144:0/668598521 >> 172.21.15.144:0/668598521 conn(0x7f67080b2fc0 msgr2=0x7f67080718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 -- 172.21.15.144:0/668598521 shutdown_connections 2023-11-28T11:53:38.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.507+0000 7f670fb5a700 1 -- 172.21.15.144:0/668598521 wait complete. 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.508+0000 7f670fb5a700 1 Processor -- start 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.508+0000 7f670fb5a700 1 -- start start 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f670806a7b0 0x7f67080c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6708071fb0 0x7f67081575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f670806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670815b230 con 0x7f6708159050 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708159960 con 0x7f670806a7b0 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6708157ae0 con 0x7f6708071fb0 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f670806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f670806d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37994/0 (socket says 172.21.15.144:37994) 2023-11-28T11:53:38.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 -- 172.21.15.144:0/790525344 learned_addr learned my addr 172.21.15.144:0/790525344 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670d0f5700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6708071fb0 0x7f67081575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670d8f6700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f670806a7b0 0x7f67080c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 -- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6708071fb0 msgr2=0x7f67081575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6708071fb0 0x7f67081575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 -- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f670806a7b0 msgr2=0x7f67080c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f670806a7b0 0x7f67080c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 -- 172.21.15.144:0/790525344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8009550 con 0x7f6708159050 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670d0f5700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6708071fb0 0x7f67081575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670e0f7700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f670806d8c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6700007db0 tx=0x7f670000d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700017070 con 0x7f6708159050 2023-11-28T11:53:38.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670000a2e0 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67080cc150 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000f040 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.509+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67080cbe70 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.511+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67080cc430 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.511+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f670000d9b0 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.511+0000 7f66fe7fc700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f66ec0682f0 0x7f66ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.511+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6700085c10 con 0x7f6708159050 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.511+0000 7f670d8f6700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f66ec0682f0 0x7f66ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.511+0000 7f670d8f6700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f66ec0682f0 0x7f66ec06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66f8014040 tx=0x7f66f8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.515+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6700052430 con 0x7f6708159050 2023-11-28T11:53:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.695+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67080c2aa0 con 0x7f6708159050 2023-11-28T11:53:38.696 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.695+0000 7f66fe7fc700 1 -- 172.21.15.144:0/790525344 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6700055a50 con 0x7f6708159050 2023-11-28T11:53:38.696 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:53:38.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f66ec0682f0 msgr2=0x7f66ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f66ec0682f0 0x7f66ec06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66f8014040 tx=0x7f66f8008030 comp rx=0 tx=0).stop 2023-11-28T11:53:38.698 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 msgr2=0x7f670806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f670806d8c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6700007db0 tx=0x7f670000d3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 shutdown_connections 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6708159050 0x7f670806d8c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f66ec0682f0 0x7f66ec06a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f670806a7b0 0x7f67080c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 --2- 172.21.15.144:0/790525344 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6708071fb0 0x7f67081575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 >> 172.21.15.144:0/790525344 conn(0x7f67080b2fc0 msgr2=0x7f67080cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 shutdown_connections 2023-11-28T11:53:38.699 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.697+0000 7f670fb5a700 1 -- 172.21.15.144:0/790525344 wait complete. 2023-11-28T11:53:38.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:38.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.831+0000 7fef2f0ae700 1 Processor -- start 2023-11-28T11:53:38.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.831+0000 7fef2f0ae700 1 -- start start 2023-11-28T11:53:38.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.831+0000 7fef2f0ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 0x7fef2806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.831+0000 7fef2f0ae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2815d060 con 0x7fef28071040 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.831+0000 7fef2f0ae700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2815d340 con 0x7fef28160540 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.831+0000 7fef2f0ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2815d620 con 0x7fef2806dee0 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef2ce4a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fef28071040 0x7fef28071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54710/0 (socket says 172.21.15.144:54710) 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3872021941 learned_addr learned my addr 172.21.15.144:0/3872021941 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef27fff700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 0x7fef2806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508150163 0 0) 0x7fef2815d060 con 0x7fef28071040 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef180035b0 con 0x7fef28071040 2023-11-28T11:53:38.836 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103698006 0 0) 0x7fef2815d340 con 0x7fef28160540 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef2815d060 con 0x7fef28160540 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781673484 0 0) 0x7fef180035b0 con 0x7fef28071040 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 msgr2=0x7fef2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 0x7fef2806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 >> v1:172.21.15.177:6789/0 conn(0x7fef28160540 legacy=0x7fef281609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2815d900 con 0x7fef28071040 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fef1c001670 con 0x7fef28071040 2023-11-28T11:53:38.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 >> v1:172.21.15.144:6789/0 conn(0x7fef28071040 legacy=0x7fef28071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef18003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef18004850 0x7fef18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 0x7fef180052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2815d620 con 0x7fef18003bc0 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1800bba0 con 0x7fef18004850 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1800bd60 con 0x7fef2806dee0 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef18003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef27fff700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef18004850 0x7fef18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.832+0000 7fef2d64b700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 0x7fef180052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 msgr2=0x7fef180052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef2806dee0 0x7fef180052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef18004850 msgr2=0x7fef18004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef18004850 0x7fef18004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3872021941 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180035b0 con 0x7fef18003bc0 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef18003fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fef1c001eb0 tx=0x7fef18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09efef56a7015d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00cd80 con 0x7fef18003bc0 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c001af0 con 0x7fef18003bc0 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef277fe700 1 -- 172.21.15.144:0/3872021941 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c005ba0 con 0x7fef18003bc0 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3872021941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 msgr2=0x7fef18003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3872021941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef18003fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fef1c001eb0 tx=0x7fef18005880 comp rx=0 tx=0).stop 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2d64b700 1 -- 172.21.15.144:0/3872021941 reap_dead start 2023-11-28T11:53:38.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3872021941 shutdown_connections 2023-11-28T11:53:38.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3872021941 >> 172.21.15.144:0/3872021941 conn(0x7fef280d0000 msgr2=0x7fef280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:38.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3872021941 shutdown_connections 2023-11-28T11:53:38.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.833+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3872021941 wait complete. 2023-11-28T11:53:38.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.835+0000 7fef2f0ae700 1 Processor -- start 2023-11-28T11:53:38.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.835+0000 7fef2f0ae700 1 -- start start 2023-11-28T11:53:38.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.835+0000 7fef2f0ae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef281f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.835+0000 7fef2f0ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef280b2e30 0x7fef280b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2f0ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef280c5e50 0x7fef280c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2f0ae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280c6810 con 0x7fef18003bc0 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2f0ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280b3880 con 0x7fef280c5e50 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2f0ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280b3b60 con 0x7fef280b2e30 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef281f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef281f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38010/0 (socket says 172.21.15.144:38010) 2023-11-28T11:53:38.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3588281675 learned_addr learned my addr 172.21.15.144:0/3588281675 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef27fff700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef280b2e30 0x7fef280b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2d64b700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef280c5e50 0x7fef280c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef280b2e30 msgr2=0x7fef280b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef280b2e30 0x7fef280b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef280c5e50 msgr2=0x7fef280c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef280c5e50 0x7fef280c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 -- 172.21.15.144:0/3588281675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2806a270 con 0x7fef18003bc0 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef27fff700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef280b2e30 0x7fef280b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2d64b700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef280c5e50 0x7fef280c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2ce4a700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef281f1d40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000c00 tx=0x7fef1c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0072d0 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c007430 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0077e0 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2815ccd0 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.836+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef280b4120 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.837+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef280c3af0 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.838+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef1c024030 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.838+0000 7fef257fa700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fef0c0682f0 0x7fef0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.838+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fef1c086ba0 con 0x7fef18003bc0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.838+0000 7fef27fff700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fef0c0682f0 0x7fef0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:38.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.838+0000 7fef27fff700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fef0c0682f0 0x7fef0c06a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef2815c910 tx=0x7fef14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:38.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:38.842+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef1c0533c0 con 0x7fef18003bc0 2023-11-28T11:53:39.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.066+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef2806a400 con 0x7fef18003bc0 2023-11-28T11:53:39.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.067+0000 7fef257fa700 1 -- 172.21.15.144:0/3588281675 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fef2806a400 con 0x7fef18003bc0 2023-11-28T11:53:39.067 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:39.068 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:53:39.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fef0c0682f0 msgr2=0x7fef0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fef0c0682f0 0x7fef0c06a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef2815c910 tx=0x7fef14008040 comp rx=0 tx=0).stop 2023-11-28T11:53:39.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 msgr2=0x7fef281f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef281f1d40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fef1c000c00 tx=0x7fef1c001980 comp rx=0 tx=0).stop 2023-11-28T11:53:39.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 shutdown_connections 2023-11-28T11:53:39.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef18003bc0 0x7fef281f1d40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fef0c0682f0 0x7fef0c06a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef280c5e50 0x7fef280c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 --2- 172.21.15.144:0/3588281675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef280b2e30 0x7fef280b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 >> 172.21.15.144:0/3588281675 conn(0x7fef280d0000 msgr2=0x7fef2815e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 shutdown_connections 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.069+0000 7fef2f0ae700 1 -- 172.21.15.144:0/3588281675 wait complete. 2023-11-28T11:53:39.071 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:39.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:39.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.201+0000 7fa857d8f700 1 Processor -- start 2023-11-28T11:53:39.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.202+0000 7fa857d8f700 1 -- start start 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.202+0000 7fa857d8f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa850140950 0x7fa8500b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.202+0000 7fa857d8f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85013ff00 con 0x7fa8500b2e70 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.202+0000 7fa857d8f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8501404c0 con 0x7fa8500cd8c0 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.202+0000 7fa857d8f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa850140dc0 con 0x7fa850140950 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85632c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa8500b2e70 0x7fa85006f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54734/0 (socket says 172.21.15.144:54734) 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85632c700 1 -- 172.21.15.144:0/3782283526 learned_addr learned my addr 172.21.15.144:0/3782283526 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:39.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85532a700 1 --2- 172.21.15.144:0/3782283526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa850140950 0x7fa8500b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa854b29700 1 -- 172.21.15.144:0/3782283526 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540320871 0 0) 0x7fa85013ff00 con 0x7fa8500b2e70 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa854b29700 1 -- 172.21.15.144:0/3782283526 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa83c0035b0 con 0x7fa8500b2e70 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa854b29700 1 -- 172.21.15.144:0/3782283526 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230551099 0 0) 0x7fa8501404c0 con 0x7fa8500cd8c0 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85532a700 1 -- 172.21.15.144:0/3782283526 >> v1:172.21.15.177:6789/0 conn(0x7fa8500cd8c0 legacy=0x7fa8500cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85532a700 1 -- 172.21.15.144:0/3782283526 >> v1:172.21.15.144:6789/0 conn(0x7fa8500b2e70 legacy=0x7fa85006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85532a700 1 -- 172.21.15.144:0/3782283526 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85015aa80 con 0x7fa850140950 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa85532a700 1 --2- 172.21.15.144:0/3782283526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa850140950 0x7fa8500b2930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009870 tx=0x7fa84c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b7fbcfed0b98a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa854b29700 1 -- 172.21.15.144:0/3782283526 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c011830 con 0x7fa850140950 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa854b29700 1 -- 172.21.15.144:0/3782283526 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c011990 con 0x7fa850140950 2023-11-28T11:53:39.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.203+0000 7fa854b29700 1 -- 172.21.15.144:0/3782283526 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c011d20 con 0x7fa850140950 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 -- 172.21.15.144:0/3782283526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa850140950 msgr2=0x7fa8500b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 --2- 172.21.15.144:0/3782283526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa850140950 0x7fa8500b2930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009870 tx=0x7fa84c009b80 comp rx=0 tx=0).stop 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 -- 172.21.15.144:0/3782283526 shutdown_connections 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 --2- 172.21.15.144:0/3782283526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa850140950 0x7fa8500b2930 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 -- 172.21.15.144:0/3782283526 >> 172.21.15.144:0/3782283526 conn(0x7fa85006aed0 msgr2=0x7fa85006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 -- 172.21.15.144:0/3782283526 shutdown_connections 2023-11-28T11:53:39.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.204+0000 7fa857d8f700 1 -- 172.21.15.144:0/3782283526 wait complete. 2023-11-28T11:53:39.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.205+0000 7fa857d8f700 1 Processor -- start 2023-11-28T11:53:39.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.205+0000 7fa857d8f700 1 -- start start 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa857d8f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500b2e70 0x7fa850154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa857d8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 0x7fa8501e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa857d8f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850140950 0x7fa8500b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa857d8f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501e50b0 con 0x7fa8500cd8c0 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa857d8f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500b4ae0 con 0x7fa850140950 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa857d8f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8501556c0 con 0x7fa8500b2e70 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 0x7fa8501e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 0x7fa8501e4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38026/0 (socket says 172.21.15.144:38026) 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 -- 172.21.15.144:0/1214898170 learned_addr learned my addr 172.21.15.144:0/1214898170 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85632c700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850140950 0x7fa8500b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa855b2b700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500b2e70 0x7fa850154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 -- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500b2e70 msgr2=0x7fa850154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500b2e70 0x7fa850154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 -- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850140950 msgr2=0x7fa8500b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850140950 0x7fa8500b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 -- 172.21.15.144:0/1214898170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0094a0 con 0x7fa8500cd8c0 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.206+0000 7fa85532a700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 0x7fa8501e4b70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009bc0 tx=0x7fa84c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.207+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01a910 con 0x7fa8500cd8c0 2023-11-28T11:53:39.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.207+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c01aa70 con 0x7fa8500cd8c0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.207+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8500b35f0 con 0x7fa8500cd8c0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.207+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01ae00 con 0x7fa8500cd8c0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.207+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8500b3310 con 0x7fa8500cd8c0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.207+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa850140150 con 0x7fa8500cd8c0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.208+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa84c02b030 con 0x7fa8500cd8c0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.208+0000 7fa8467fc700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa8340682f0 0x7fa83406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.208+0000 7fa855b2b700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa8340682f0 0x7fa83406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.208+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa84c086ef0 con 0x7fa8500cd8c0 2023-11-28T11:53:39.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.209+0000 7fa855b2b700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa8340682f0 0x7fa83406a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa840000fb0 tx=0x7fa840009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.213+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa84c053710 con 0x7fa8500cd8c0 2023-11-28T11:53:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.396+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa850068690 con 0x7fa8500cd8c0 2023-11-28T11:53:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.397+0000 7fa8467fc700 1 -- 172.21.15.144:0/1214898170 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18009 (secure 0 0 0) 0x7fa84c015090 con 0x7fa8500cd8c0 2023-11-28T11:53:39.398 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:39.399 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":48,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:05.732387+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa8340682f0 msgr2=0x7fa83406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa8340682f0 0x7fa83406a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa840000fb0 tx=0x7fa840009040 comp rx=0 tx=0).stop 2023-11-28T11:53:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 msgr2=0x7fa8501e4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 0x7fa8501e4b70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009bc0 tx=0x7fa84c004750 comp rx=0 tx=0).stop 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 shutdown_connections 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa8340682f0 0x7fa83406a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa850140950 0x7fa8500b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8500b2e70 0x7fa850154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 --2- 172.21.15.144:0/1214898170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8500cd8c0 0x7fa8501e4b70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 >> 172.21.15.144:0/1214898170 conn(0x7fa85006aed0 msgr2=0x7fa85013db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 shutdown_connections 2023-11-28T11:53:39.401 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.399+0000 7fa857d8f700 1 -- 172.21.15.144:0/1214898170 wait complete. 2023-11-28T11:53:39.412 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:53:39.413 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:39.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 Processor -- start 2023-11-28T11:53:39.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 -- start start 2023-11-28T11:53:39.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff31c070c50 0x7ff31c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c0b0bb0 0x7ff31c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c15e1a0 con 0x7ff31c158200 2023-11-28T11:53:39.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c15f070 con 0x7ff31c070c50 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.532+0000 7ff321474700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0b0fd0 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff31c070c50 0x7ff31c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31b7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c0b0bb0 0x7ff31c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31affd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7ff31c158200 0x7ff31c0caa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55234/0 (socket says 172.21.15.144:55234) 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31affd700 1 -- 172.21.15.144:0/1813660817 learned_addr learned my addr 172.21.15.144:0/1813660817 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff31c070c50 0x7ff31c0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:40308/0 (socket says 172.21.15.144:40308) 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 -- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c0b0bb0 msgr2=0x7ff31c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 --2- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c0b0bb0 0x7ff31c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 -- 172.21.15.144:0/1813660817 >> v1:172.21.15.177:6789/0 conn(0x7ff31c158200 legacy=0x7ff31c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff319ffb700 1 -- 172.21.15.144:0/1813660817 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086702094 0 0) 0x7ff31c15e1a0 con 0x7ff31c158200 2023-11-28T11:53:39.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 -- 172.21.15.144:0/1813660817 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0cb180 con 0x7ff31c070c50 2023-11-28T11:53:39.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff31a7fc700 1 --2- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c070c50 0x7ff31c0b0670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff310009840 tx=0x7ff310009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39047b8dc6861ed5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.533+0000 7ff319ffb700 1 -- 172.21.15.144:0/1813660817 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310011830 con 0x7ff31c070c50 2023-11-28T11:53:39.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff319ffb700 1 -- 172.21.15.144:0/1813660817 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff310011990 con 0x7ff31c070c50 2023-11-28T11:53:39.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff319ffb700 1 -- 172.21.15.144:0/1813660817 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310011d20 con 0x7ff31c070c50 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 -- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c070c50 msgr2=0x7ff31c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 --2- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c070c50 0x7ff31c0b0670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff310009840 tx=0x7ff310009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 -- 172.21.15.144:0/1813660817 shutdown_connections 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 --2- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c0b0bb0 0x7ff31c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 --2- 172.21.15.144:0/1813660817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c070c50 0x7ff31c0b0670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 -- 172.21.15.144:0/1813660817 >> 172.21.15.144:0/1813660817 conn(0x7ff31c0cb670 msgr2=0x7ff31c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:39.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 -- 172.21.15.144:0/1813660817 shutdown_connections 2023-11-28T11:53:39.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.534+0000 7ff321474700 1 -- 172.21.15.144:0/1813660817 wait complete. 2023-11-28T11:53:39.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.536+0000 7ff321474700 1 Processor -- start 2023-11-28T11:53:39.538 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.536+0000 7ff321474700 1 -- start start 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff321474700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c070c50 0x7ff31c1f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff321474700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 0x7ff31c1f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff321474700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c158200 0x7ff31c0aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff321474700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c1f3f70 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff321474700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c1f4250 con 0x7ff31c158200 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff321474700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c1f4530 con 0x7ff31c070c50 2023-11-28T11:53:39.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 0x7ff31c1f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 0x7ff31c1f09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38040/0 (socket says 172.21.15.144:38040) 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 -- 172.21.15.144:0/180988277 learned_addr learned my addr 172.21.15.144:0/180988277 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31affd700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c070c50 0x7ff31c1f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31b7fe700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c158200 0x7ff31c0aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 -- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c070c50 msgr2=0x7ff31c1f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c070c50 0x7ff31c1f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 -- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c158200 msgr2=0x7ff31c0aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c158200 0x7ff31c0aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 -- 172.21.15.144:0/180988277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100094a0 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.537+0000 7ff31a7fc700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 0x7ff31c1f09b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff310009fd0 tx=0x7ff310004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.538+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100036a0 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.538+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31001f070 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.538+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c1585d0 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.538+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310003a60 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.538+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c0710f0 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.539+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c15e3f0 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.539+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff31000f050 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.539+0000 7ff3037fe700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3080682f0 0x7ff30806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.539+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff310021070 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.539+0000 7ff31affd700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3080682f0 0x7ff30806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.540+0000 7ff31affd700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3080682f0 0x7ff30806a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff30c000f60 tx=0x7ff30c007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.544+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff310053e60 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.771+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff31c06fd60 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.772+0000 7ff3037fe700 1 -- 172.21.15.144:0/180988277 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7ff310057480 con 0x7ff31c0b0bb0 2023-11-28T11:53:39.773 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:39.773 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:53:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3080682f0 msgr2=0x7ff30806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3080682f0 0x7ff30806a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff30c000f60 tx=0x7ff30c007c90 comp rx=0 tx=0).stop 2023-11-28T11:53:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 msgr2=0x7ff31c1f09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 0x7ff31c1f09b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff310009fd0 tx=0x7ff310004750 comp rx=0 tx=0).stop 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 shutdown_connections 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff3080682f0 0x7ff30806a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff31c158200 0x7ff31c0aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff31c070c50 0x7ff31c1f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 --2- 172.21.15.144:0/180988277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff31c0b0bb0 0x7ff31c1f09b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 >> 172.21.15.144:0/180988277 conn(0x7ff31c0cb670 msgr2=0x7ff31c0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 shutdown_connections 2023-11-28T11:53:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.774+0000 7ff321474700 1 -- 172.21.15.144:0/180988277 wait complete. 2023-11-28T11:53:39.777 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:39.787 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:39.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.906+0000 7fb54bd09700 1 Processor -- start 2023-11-28T11:53:39.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.907+0000 7fb54bd09700 1 -- start start 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.907+0000 7fb54bd09700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb5440cd8c0 0x7fb544140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.907+0000 7fb54bd09700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 0x7fb54406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.907+0000 7fb54bd09700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54413ff00 con 0x7fb5440cdc90 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.907+0000 7fb54bd09700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5441404c0 con 0x7fb5440cd8c0 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.907+0000 7fb54bd09700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54415aa80 con 0x7fb54406f850 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb54a2a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 0x7fb54406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb549aa5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb5440cd8c0 0x7fb544140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb54a2a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 0x7fb54406fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:50130/0 (socket says 172.21.15.144:50130) 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb54a2a6700 1 -- 172.21.15.144:0/2397512597 learned_addr learned my addr 172.21.15.144:0/2397512597 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb5492a4700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fb5440cdc90 0x7fb5440b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55248/0 (socket says 172.21.15.144:55248) 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb549aa5700 1 -- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 msgr2=0x7fb54406fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb549aa5700 1 --2- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 0x7fb54406fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb549aa5700 1 -- 172.21.15.144:0/2397512597 >> v1:172.21.15.177:6789/0 conn(0x7fb5440cdc90 legacy=0x7fb5440b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb548aa3700 1 -- 172.21.15.144:0/2397512597 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794425784 0 0) 0x7fb54413ff00 con 0x7fb5440cdc90 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb549aa5700 1 -- 172.21.15.144:0/2397512597 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440b3040 con 0x7fb5440cd8c0 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb54a2a6700 1 --2- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 0x7fb54406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb549aa5700 1 --2- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5440cd8c0 0x7fb544140950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb534009820 tx=0x7fb534009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883a909e58f0fef3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb548aa3700 1 -- 172.21.15.144:0/2397512597 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534005e30 con 0x7fb5440cd8c0 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb548aa3700 1 -- 172.21.15.144:0/2397512597 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534006030 con 0x7fb5440cd8c0 2023-11-28T11:53:39.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.908+0000 7fb548aa3700 1 -- 172.21.15.144:0/2397512597 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340063c0 con 0x7fb5440cd8c0 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 -- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5440cd8c0 msgr2=0x7fb544140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 --2- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5440cd8c0 0x7fb544140950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb534009820 tx=0x7fb534009b30 comp rx=0 tx=0).stop 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 -- 172.21.15.144:0/2397512597 shutdown_connections 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 --2- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb54406f850 0x7fb54406fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 --2- 172.21.15.144:0/2397512597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5440cd8c0 0x7fb544140950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 -- 172.21.15.144:0/2397512597 >> 172.21.15.144:0/2397512597 conn(0x7fb54406aed0 msgr2=0x7fb54406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 -- 172.21.15.144:0/2397512597 shutdown_connections 2023-11-28T11:53:39.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.909+0000 7fb54bd09700 1 -- 172.21.15.144:0/2397512597 wait complete. 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.911+0000 7fb54bd09700 1 Processor -- start 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.911+0000 7fb54bd09700 1 -- start start 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb54406f850 0x7fb544149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5440cd8c0 0x7fb5441f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 0x7fb5440c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5440c49e0 con 0x7fb5440cdc90 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb544149940 con 0x7fb54406f850 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5440c4bf0 con 0x7fb5440cd8c0 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 0x7fb5440c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 0x7fb5440c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38054/0 (socket says 172.21.15.144:38054) 2023-11-28T11:53:39.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 -- 172.21.15.144:0/120301803 learned_addr learned my addr 172.21.15.144:0/120301803 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:39.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb549aa5700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb54406f850 0x7fb544149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb5492a4700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5440cd8c0 0x7fb5441f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 -- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5440cd8c0 msgr2=0x7fb5441f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5440cd8c0 0x7fb5441f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 -- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb54406f850 msgr2=0x7fb544149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb54406f850 0x7fb544149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 -- 172.21.15.144:0/120301803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c009b30 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54a2a6700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 0x7fb5440c70b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00efc0 tx=0x7fb53c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c018070 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c00f040 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb534009430 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003c20 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.912+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5440b35f0 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.913+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5440b3310 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.914+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb53c00c880 con 0x7fb5440cdc90 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.914+0000 7fb53a7fc700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb5280682f0 0x7fb52806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:39.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.914+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb53c085f50 con 0x7fb5440cdc90 2023-11-28T11:53:39.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.914+0000 7fb549aa5700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb5280682f0 0x7fb52806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:39.918 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.914+0000 7fb549aa5700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb5280682f0 0x7fb52806a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb534000c00 tx=0x7fb534008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:39.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:39.918+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb53c052770 con 0x7fb5440cdc90 2023-11-28T11:53:40.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.145+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb544140150 con 0x7fb5440cdc90 2023-11-28T11:53:40.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.146+0000 7fb53a7fc700 1 -- 172.21.15.144:0/120301803 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4660 (secure 0 0 0) 0x7fb53c055d90 con 0x7fb5440cdc90 2023-11-28T11:53:40.147 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:40.147 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:52:50.207501+0000","modified":"2023-11-28T11:52:53.224270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5535":{"gid":5535,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6834/1698439070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1698439070},{"type":"v1","addr":"172.21.15.177:6834","nonce":1698439070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb5280682f0 msgr2=0x7fb52806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb5280682f0 0x7fb52806a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb534000c00 tx=0x7fb534008070 comp rx=0 tx=0).stop 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 msgr2=0x7fb5440c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 0x7fb5440c70b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00efc0 tx=0x7fb53c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 shutdown_connections 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5440cdc90 0x7fb5440c70b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb5280682f0 0x7fb52806a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb54406f850 0x7fb544149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 --2- 172.21.15.144:0/120301803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb5440cd8c0 0x7fb5441f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 >> 172.21.15.144:0/120301803 conn(0x7fb54406aed0 msgr2=0x7fb5440b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 shutdown_connections 2023-11-28T11:53:40.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.148+0000 7fb54bd09700 1 -- 172.21.15.144:0/120301803 wait complete. 2023-11-28T11:53:40.150 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2023-11-28T11:53:40.160 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:40.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.279+0000 7f16bcde9700 1 Processor -- start 2023-11-28T11:53:40.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.279+0000 7f16bcde9700 1 -- start start 2023-11-28T11:53:40.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16bcde9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16b806dee0 0x7f16b806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16bcde9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f16b8160540 0x7f16b8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16bcde9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b815d060 con 0x7f16b8071040 2023-11-28T11:53:40.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16bcde9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b815d340 con 0x7f16b806dee0 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16bcde9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b815d620 con 0x7f16b8160540 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16b5d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16b806dee0 0x7f16b806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16b659c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f16b8071040 0x7f16b8071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54770/0 (socket says 172.21.15.144:54770) 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16b659c700 1 -- 172.21.15.144:0/1460464828 learned_addr learned my addr 172.21.15.144:0/1460464828 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.280+0000 7f16b6d9d700 1 --2- 172.21.15.144:0/1460464828 >> v2:172.21.15.177:3300/0 conn(0x7f16b8160540 0x7f16b8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1460464828 >> v2:172.21.15.177:3300/0 conn(0x7f16b8160540 msgr2=0x7f16b8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b5d9b700 1 --2- 172.21.15.144:0/1460464828 >> v2:172.21.15.177:3300/0 conn(0x7f16b8160540 0x7f16b8160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1460464828 >> v1:172.21.15.144:6789/0 conn(0x7f16b8071040 legacy=0x7f16b8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b559a700 1 -- 172.21.15.144:0/1460464828 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292935113 0 0) 0x7f16b815d060 con 0x7f16b8071040 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1460464828 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b815d900 con 0x7f16b806dee0 2023-11-28T11:53:40.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b6d9d700 1 --2- 172.21.15.144:0/1460464828 >> v2:172.21.15.177:3300/0 conn(0x7f16b8160540 0x7f16b8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b5d9b700 1 --2- 172.21.15.144:0/1460464828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b806dee0 0x7f16b806e330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f16b80c3a10 tx=0x7f16a8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be7ae2c9f19be49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b559a700 1 -- 172.21.15.144:0/1460464828 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011830 con 0x7f16b806dee0 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b559a700 1 -- 172.21.15.144:0/1460464828 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8011990 con 0x7f16b806dee0 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16b559a700 1 -- 172.21.15.144:0/1460464828 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011d20 con 0x7f16b806dee0 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16bcde9700 1 -- 172.21.15.144:0/1460464828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b806dee0 msgr2=0x7f16b806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1460464828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b806dee0 0x7f16b806e330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f16b80c3a10 tx=0x7f16a8009b20 comp rx=0 tx=0).stop 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16bcde9700 1 -- 172.21.15.144:0/1460464828 shutdown_connections 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1460464828 >> v2:172.21.15.177:3300/0 conn(0x7f16b8160540 0x7f16b8160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1460464828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b806dee0 0x7f16b806e330 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.281+0000 7f16bcde9700 1 -- 172.21.15.144:0/1460464828 >> 172.21.15.144:0/1460464828 conn(0x7f16b80d0000 msgr2=0x7f16b80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:40.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.282+0000 7f16bcde9700 1 -- 172.21.15.144:0/1460464828 shutdown_connections 2023-11-28T11:53:40.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.282+0000 7f16bcde9700 1 -- 172.21.15.144:0/1460464828 wait complete. 2023-11-28T11:53:40.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.283+0000 7f16bcde9700 1 Processor -- start 2023-11-28T11:53:40.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.283+0000 7f16bcde9700 1 -- start start 2023-11-28T11:53:40.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16b806dee0 0x7f16b814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 0x7f16b80c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b8160540 0x7f16b80b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b814ba00 con 0x7f16b8071040 2023-11-28T11:53:40.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80c4db0 con 0x7f16b8160540 2023-11-28T11:53:40.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80c5090 con 0x7f16b806dee0 2023-11-28T11:53:40.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 0x7f16b80c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 0x7f16b80c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38066/0 (socket says 172.21.15.144:38066) 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1740588568 learned_addr learned my addr 172.21.15.144:0/1740588568 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b6d9d700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b8160540 0x7f16b80b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b659c700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16b806dee0 0x7f16b814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16b806dee0 msgr2=0x7f16b814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16b806dee0 0x7f16b814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b8160540 msgr2=0x7f16b80b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b8160540 0x7f16b80b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 -- 172.21.15.144:0/1740588568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80094a0 con 0x7f16b8071040 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b6d9d700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b8160540 0x7f16b80b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16b5d9b700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 0x7f16b80c44b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f16a8014040 tx=0x7f16a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80036a0 con 0x7f16b8071040 2023-11-28T11:53:40.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b815ccd0 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a801e070 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8003a40 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.284+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b806e870 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.285+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b806a270 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.286+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f16a800f050 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.286+0000 7f16a6ffd700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16980682f0 0x7f169806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.286+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f16a8086d50 con 0x7f16b8071040 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.286+0000 7f16b659c700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16980682f0 0x7f169806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.287+0000 7f16b659c700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16980682f0 0x7f169806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000f60 tx=0x7f16ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:40.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.291+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f16a80534f0 con 0x7f16b8071040 2023-11-28T11:53:40.475 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.474+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16b80720a0 con 0x7f16b8071040 2023-11-28T11:53:40.476 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.475+0000 7f16a6ffd700 1 -- 172.21.15.144:0/1740588568 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+18009 (secure 0 0 0) 0x7f16a8015020 con 0x7f16b8071040 2023-11-28T11:53:40.476 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:40.477 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":48,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:05.732387+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:52:48.228399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T11:52:49.180034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.21","mappings":[{"from":0,"to":6}]},{"pgid":"8.2a","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16980682f0 msgr2=0x7f169806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16980682f0 0x7f169806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000f60 tx=0x7f16ac009040 comp rx=0 tx=0).stop 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 msgr2=0x7f16b80c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 0x7f16b80c44b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f16a8014040 tx=0x7f16a8004750 comp rx=0 tx=0).stop 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 shutdown_connections 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f16980682f0 0x7f169806a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.479 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16b8160540 0x7f16b80b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16b806dee0 0x7f16b814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 --2- 172.21.15.144:0/1740588568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b8071040 0x7f16b80c44b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 >> 172.21.15.144:0/1740588568 conn(0x7f16b80d0000 msgr2=0x7f16b815e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:40.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 shutdown_connections 2023-11-28T11:53:40.480 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.478+0000 7f16bcde9700 1 -- 172.21.15.144:0/1740588568 wait complete. 2023-11-28T11:53:40.491 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:53:40.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.610+0000 7fb26a8e8700 1 Processor -- start 2023-11-28T11:53:40.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.610+0000 7fb26a8e8700 1 -- start start 2023-11-28T11:53:40.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb26a8e8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2640cd8c0 0x7fb264140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb26a8e8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb2640cdc90 0x7fb2640b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb26a8e8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26413ff00 con 0x7fb2640b2e70 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb26a8e8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2641404c0 con 0x7fb2640cd8c0 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb26a8e8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26415aa80 con 0x7fb2640cdc90 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2640cd8c0 0x7fb264140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2640cd8c0 0x7fb264140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38072/0 (socket says 172.21.15.144:38072) 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 -- 172.21.15.144:0/498745027 learned_addr learned my addr 172.21.15.144:0/498745027 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:40.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 -- 172.21.15.144:0/498745027 >> v2:172.21.15.177:3300/0 conn(0x7fb2640cdc90 msgr2=0x7fb2640b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:40.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb2637fe700 1 --2- 172.21.15.144:0/498745027 >> v2:172.21.15.177:3300/0 conn(0x7fb2640cdc90 0x7fb2640b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 --2- 172.21.15.144:0/498745027 >> v2:172.21.15.177:3300/0 conn(0x7fb2640cdc90 0x7fb2640b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 -- 172.21.15.144:0/498745027 >> v1:172.21.15.177:6789/0 conn(0x7fb2640b2e70 legacy=0x7fb26406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 -- 172.21.15.144:0/498745027 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2640b3310 con 0x7fb2640cd8c0 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb2637fe700 1 --2- 172.21.15.144:0/498745027 >> v2:172.21.15.177:3300/0 conn(0x7fb2640cdc90 0x7fb2640b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.611+0000 7fb263fff700 1 --2- 172.21.15.144:0/498745027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640cd8c0 0x7fb264140950 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb264071c40 tx=0x7fb258009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c4656ac0aedf23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb262ffd700 1 -- 172.21.15.144:0/498745027 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258005e30 con 0x7fb2640cd8c0 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb262ffd700 1 -- 172.21.15.144:0/498745027 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258006030 con 0x7fb2640cd8c0 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb262ffd700 1 -- 172.21.15.144:0/498745027 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580063e0 con 0x7fb2640cd8c0 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 -- 172.21.15.144:0/498745027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640cd8c0 msgr2=0x7fb264140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/498745027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640cd8c0 0x7fb264140950 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb264071c40 tx=0x7fb258009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 -- 172.21.15.144:0/498745027 shutdown_connections 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/498745027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640cd8c0 0x7fb264140950 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/498745027 >> v2:172.21.15.177:3300/0 conn(0x7fb2640cdc90 0x7fb2640b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 -- 172.21.15.144:0/498745027 >> 172.21.15.144:0/498745027 conn(0x7fb26406aed0 msgr2=0x7fb26406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:40.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 -- 172.21.15.144:0/498745027 shutdown_connections 2023-11-28T11:53:40.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.612+0000 7fb26a8e8700 1 -- 172.21.15.144:0/498745027 wait complete. 2023-11-28T11:53:40.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.614+0000 7fb26a8e8700 1 Processor -- start 2023-11-28T11:53:40.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.614+0000 7fb26a8e8700 1 -- start start 2023-11-28T11:53:40.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 0x7fb26415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2640cd8c0 0x7fb2640c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2640cdc90 0x7fb2640c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26415c2b0 con 0x7fb2640b2e70 2023-11-28T11:53:40.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2641f1270 con 0x7fb2640cdc90 2023-11-28T11:53:40.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26415b950 con 0x7fb2640cd8c0 2023-11-28T11:53:40.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 0x7fb26415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 0x7fb26415c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38078/0 (socket says 172.21.15.144:38078) 2023-11-28T11:53:40.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 -- 172.21.15.144:0/2665093861 learned_addr learned my addr 172.21.15.144:0/2665093861 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:40.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb268e85700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2640cdc90 0x7fb2640c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb2637fe700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2640cd8c0 0x7fb2640c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 -- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2640cd8c0 msgr2=0x7fb2640c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2640cd8c0 0x7fb2640c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 -- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2640cdc90 msgr2=0x7fb2640c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2640cdc90 0x7fb2640c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 -- 172.21.15.144:0/2665093861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258009500 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb268e85700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2640cdc90 0x7fb2640c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb2637fe700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2640cd8c0 0x7fb2640c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb263fff700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 0x7fb26415c6d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb258006000 tx=0x7fb258004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258025480 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb258020070 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb264140150 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.615+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258025800 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.616+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2640b35f0 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.616+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2640ce100 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.617+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb25800f050 con 0x7fb2640b2e70 2023-11-28T11:53:40.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.617+0000 7fb260ff9700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2500682f0 0x7fb25006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:40.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.617+0000 7fb2637fe700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2500682f0 0x7fb25006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:40.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.617+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb2580875b0 con 0x7fb2640b2e70 2023-11-28T11:53:40.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.618+0000 7fb2637fe700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2500682f0 0x7fb25006a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb26415e0c0 tx=0x7fb254008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:40.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.622+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb258053dd0 con 0x7fb2640b2e70 2023-11-28T11:53:40.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:40.848+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb26415c520 con 0x7fb2640b2e70 2023-11-28T11:53:41.823 INFO:tasks.ceph.mon.a.smithi144.stderr:2023-11-28T11:53:41.822+0000 7fdb2a043700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:53:41.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.833+0000 7fb260ff9700 1 -- 172.21.15.144:0/2665093861 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb2580573f0 con 0x7fb2640b2e70 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2500682f0 msgr2=0x7fb25006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2500682f0 0x7fb25006a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb26415e0c0 tx=0x7fb254008040 comp rx=0 tx=0).stop 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 msgr2=0x7fb26415c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 0x7fb26415c6d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb258006000 tx=0x7fb258004750 comp rx=0 tx=0).stop 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 shutdown_connections 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2640b2e70 0x7fb26415c6d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb2500682f0 0x7fb25006a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2640cdc90 0x7fb2640c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 --2- 172.21.15.144:0/2665093861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2640cd8c0 0x7fb2640c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 >> 172.21.15.144:0/2665093861 conn(0x7fb26406aed0 msgr2=0x7fb2640d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:41.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 shutdown_connections 2023-11-28T11:53:41.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.836+0000 7fb26a8e8700 1 -- 172.21.15.144:0/2665093861 wait complete. 2023-11-28T11:53:41.839 INFO:tasks.ceph.mds.d.smithi177.stderr: -6584> 2023-11-28T11:52:55.798+0000 7f318610b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:53:41.839 INFO:teuthology.orchestra.run.smithi144.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:53:41.842 INFO:tasks.ceph.mds.d.smithi177.stderr: -2990> 2023-11-28T11:53:03.087+0000 7f3182904700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5066=0-4194304@1} caps={5066=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5066 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55d371a67700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:53:41.851 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:53:41.870 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T11:53:41.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.979+0000 7f0a5f915700 1 Processor -- start 2023-11-28T11:53:41.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.979+0000 7f0a5f915700 1 -- start start 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5f915700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0a58140950 0x7f0a580b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5f915700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0a580b2e70 0x7f0a5806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5f915700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5813ff00 con 0x7f0a580cd8c0 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5f915700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a581404c0 con 0x7f0a58140950 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5f915700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a58140dc0 con 0x7f0a580b2e70 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5ceb0700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0a58140950 0x7f0a580b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:41.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5d6b1700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0a580cd8c0 0x7f0a580cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54798/0 (socket says 172.21.15.144:54798) 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5d6b1700 1 -- 172.21.15.144:0/1648853321 learned_addr learned my addr 172.21.15.144:0/1648853321 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5deb2700 1 --2- 172.21.15.144:0/1648853321 >> v2:172.21.15.177:3300/0 conn(0x7f0a580b2e70 0x7f0a5806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/1648853321 >> v2:172.21.15.177:3300/0 conn(0x7f0a580b2e70 msgr2=0x7f0a5806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5ceb0700 1 --2- 172.21.15.144:0/1648853321 >> v2:172.21.15.177:3300/0 conn(0x7f0a580b2e70 0x7f0a5806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/1648853321 >> v1:172.21.15.144:6789/0 conn(0x7f0a580cd8c0 legacy=0x7f0a580cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/1648853321 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5815aa80 con 0x7f0a58140950 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a4ffff700 1 -- 172.21.15.144:0/1648853321 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605951752 0 0) 0x7f0a5813ff00 con 0x7f0a580cd8c0 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a5ceb0700 1 --2- 172.21.15.144:0/1648853321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a58140950 0x7f0a580b2930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0a58144350 tx=0x7f0a48009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9adf259af6057d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.980+0000 7f0a4ffff700 1 -- 172.21.15.144:0/1648853321 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48011830 con 0x7f0a58140950 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a4ffff700 1 -- 172.21.15.144:0/1648853321 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a48011990 con 0x7f0a58140950 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a4ffff700 1 -- 172.21.15.144:0/1648853321 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48011d20 con 0x7f0a58140950 2023-11-28T11:53:41.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 -- 172.21.15.144:0/1648853321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a58140950 msgr2=0x7f0a580b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 --2- 172.21.15.144:0/1648853321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a58140950 0x7f0a580b2930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0a58144350 tx=0x7f0a48009b20 comp rx=0 tx=0).stop 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 -- 172.21.15.144:0/1648853321 shutdown_connections 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 --2- 172.21.15.144:0/1648853321 >> v2:172.21.15.177:3300/0 conn(0x7f0a580b2e70 0x7f0a5806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 --2- 172.21.15.144:0/1648853321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a58140950 0x7f0a580b2930 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 -- 172.21.15.144:0/1648853321 >> 172.21.15.144:0/1648853321 conn(0x7f0a5806aed0 msgr2=0x7f0a5806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 -- 172.21.15.144:0/1648853321 shutdown_connections 2023-11-28T11:53:41.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.981+0000 7f0a5f915700 1 -- 172.21.15.144:0/1648853321 wait complete. 2023-11-28T11:53:41.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.982+0000 7f0a5f915700 1 Processor -- start 2023-11-28T11:53:41.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.982+0000 7f0a5f915700 1 -- start start 2023-11-28T11:53:41.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5f915700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a580b2e70 0x7f0a580c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:41.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5f915700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 0x7f0a581efb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:41.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5f915700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a58140950 0x7f0a580ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5f915700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580c8fd0 con 0x7f0a580cd8c0 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5f915700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5815c2f0 con 0x7f0a580b2e70 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5f915700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a580c9270 con 0x7f0a58140950 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 0x7f0a581efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 0x7f0a581efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38094/0 (socket says 172.21.15.144:38094) 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/618649319 learned_addr learned my addr 172.21.15.144:0/618649319 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5d6b1700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a580b2e70 0x7f0a580c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5deb2700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a58140950 0x7f0a580ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a58140950 msgr2=0x7f0a580ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a58140950 0x7f0a580ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a580b2e70 msgr2=0x7f0a580c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:41.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.983+0000 7f0a5ceb0700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a580b2e70 0x7f0a580c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a5ceb0700 1 -- 172.21.15.144:0/618649319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a480094a0 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a5d6b1700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a580b2e70 0x7f0a580c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a5ceb0700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 0x7f0a581efb60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009b60 tx=0x7f0a48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480038a0 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4801e070 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a580b35f0 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48003c00 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.984+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a580b3310 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.985+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a58140150 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.985+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a4800f050 con 0x7f0a580cd8c0 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.986+0000 7f0a4dffb700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0a3c0682f0 0x7f0a3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:41.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.986+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f0a48086d10 con 0x7f0a580cd8c0 2023-11-28T11:53:41.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.986+0000 7f0a5d6b1700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0a3c0682f0 0x7f0a3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:41.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.986+0000 7f0a5d6b1700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0a3c0682f0 0x7f0a3c06a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0a540010e0 tx=0x7f0a54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:41.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:41.990+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0a480534e0 con 0x7f0a580cd8c0 2023-11-28T11:53:42.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.215+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0a58068690 con 0x7f0a580cd8c0 2023-11-28T11:53:42.835 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.834+0000 7f0a4dffb700 1 -- 172.21.15.144:0/618649319 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0a48056b00 con 0x7f0a580cd8c0 2023-11-28T11:53:42.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0a3c0682f0 msgr2=0x7f0a3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0a3c0682f0 0x7f0a3c06a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0a540010e0 tx=0x7f0a54009040 comp rx=0 tx=0).stop 2023-11-28T11:53:42.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 msgr2=0x7f0a581efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 0x7f0a581efb60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009b60 tx=0x7f0a48004750 comp rx=0 tx=0).stop 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 shutdown_connections 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0a3c0682f0 0x7f0a3c06a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0a58140950 0x7f0a580ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0a580b2e70 0x7f0a580c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.837+0000 7f0a5f915700 1 --2- 172.21.15.144:0/618649319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a580cd8c0 0x7f0a581efb60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.838+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 >> 172.21.15.144:0/618649319 conn(0x7f0a5806aed0 msgr2=0x7f0a5813ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.838+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 shutdown_connections 2023-11-28T11:53:42.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.838+0000 7f0a5f915700 1 -- 172.21.15.144:0/618649319 wait complete. 2023-11-28T11:53:42.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:53:42.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.983+0000 7f9cb1087700 1 Processor -- start 2023-11-28T11:53:42.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.983+0000 7f9cb1087700 1 -- start start 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9cb1087700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9cac071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9cb1087700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0cc7b0 con 0x7f9cac0cc260 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9cb1087700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0cd610 con 0x7f9cac0d0470 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9cb1087700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac159a80 con 0x7f9cac1596b0 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9caa59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9cac0cc260 0x7f9cac0cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54816/0 (socket says 172.21.15.144:54816) 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9caa59c700 1 -- 172.21.15.144:0/840633301 learned_addr learned my addr 172.21.15.144:0/840633301 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278090933 0 0) 0x7f9cac0cc7b0 con 0x7f9cac0cc260 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c940035b0 con 0x7f9cac0cc260 2023-11-28T11:53:42.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9caad9d700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9cac071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319024060 0 0) 0x7f9c940035b0 con 0x7f9cac0cc260 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 msgr2=0x7f9cac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9cac071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 >> v1:172.21.15.177:6789/0 conn(0x7f9cac0d0470 legacy=0x7f9cac06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac0cca00 con 0x7f9cac0cc260 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f9c9c001670 con 0x7f9cac0cc260 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 >> v1:172.21.15.144:6789/0 conn(0x7f9cac0cc260 legacy=0x7f9cac0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c94003a30 0x7f9c94003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94004850 0x7f9c94004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9c94005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac159a80 con 0x7f9c94003a30 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9400bb40 con 0x7f9c94004850 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.984+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9400bd00 con 0x7f9cac1596b0 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c94003a30 0x7f9c94003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caa59c700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94004850 0x7f9c94004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9cab59e700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9c94005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 -- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 msgr2=0x7f9c94005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9c94005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 -- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94004850 msgr2=0x7f9c94004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94004850 0x7f9c94004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 -- 172.21.15.144:0/840633301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940035b0 con 0x7f9c94003a30 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9cab59e700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1596b0 0x7f9c94005240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9caad9d700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c94003a30 0x7f9c94003e40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009a30 tx=0x7f9c94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a497139ee2b87d55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000c070 con 0x7f9c94003a30 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0009da0 con 0x7f9c94003a30 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.985+0000 7f9ca9d9b700 1 -- 172.21.15.144:0/840633301 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0010a90 con 0x7f9c94003a30 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cb1087700 1 -- 172.21.15.144:0/840633301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c94003a30 msgr2=0x7f9c94003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cb1087700 1 --2- 172.21.15.144:0/840633301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c94003a30 0x7f9c94003e40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009a30 tx=0x7f9c94005880 comp rx=0 tx=0).stop 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cab59e700 1 -- 172.21.15.144:0/840633301 reap_dead start 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cb1087700 1 -- 172.21.15.144:0/840633301 shutdown_connections 2023-11-28T11:53:42.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cb1087700 1 -- 172.21.15.144:0/840633301 >> 172.21.15.144:0/840633301 conn(0x7f9cac06b380 msgr2=0x7f9cac06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:42.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cb1087700 1 -- 172.21.15.144:0/840633301 shutdown_connections 2023-11-28T11:53:42.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.986+0000 7f9cb1087700 1 -- 172.21.15.144:0/840633301 wait complete. 2023-11-28T11:53:42.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.987+0000 7f9cb1087700 1 Processor -- start 2023-11-28T11:53:42.992 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.987+0000 7f9cb1087700 1 -- start start 2023-11-28T11:53:42.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cb1087700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94003a30 0x7f9cac1eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cb1087700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1ee1d0 0x7f9cac1f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cb1087700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 0x7f9cac1f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cb1087700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac15aa70 con 0x7f9cac1ee640 2023-11-28T11:53:42.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cb1087700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac15a440 con 0x7f9c94003a30 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cb1087700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0c9bd0 con 0x7f9cac1ee1d0 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 0x7f9cac1f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 0x7f9cac1f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38116/0 (socket says 172.21.15.144:38116) 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 -- 172.21.15.144:0/2015028591 learned_addr learned my addr 172.21.15.144:0/2015028591 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9caa59c700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1ee1d0 0x7f9cac1f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9caad9d700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94003a30 0x7f9cac1eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 -- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1ee1d0 msgr2=0x7f9cac1f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1ee1d0 0x7f9cac1f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 -- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94003a30 msgr2=0x7f9cac1eefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:42.995 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94003a30 0x7f9cac1eefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 -- 172.21.15.144:0/2015028591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac0ccfc0 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.988+0000 7f9cab59e700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 0x7f9cac1f0e50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9c940047d0 tx=0x7f9ca40123b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.989+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4003e10 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.989+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4014040 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.989+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4012aa0 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.989+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac0ccce0 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.989+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac0cd2a0 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.989+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac068690 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.990+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ca401f030 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.990+0000 7f9c937fe700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c980682f0 0x7f9c9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.990+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f9ca40894c0 con 0x7f9cac1ee640 2023-11-28T11:53:42.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.990+0000 7f9caad9d700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c980682f0 0x7f9c9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:42.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.991+0000 7f9caad9d700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c980682f0 0x7f9c9806a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0000c00 tx=0x7f9ca001e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:42.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:42.995+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ca40515e0 con 0x7f9cac1ee640 2023-11-28T11:53:43.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.176+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9cac0b2750 con 0x7f9cac1ee640 2023-11-28T11:53:43.894 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.893+0000 7f9c937fe700 1 -- 172.21.15.144:0/2015028591 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9ca4055ce0 con 0x7f9cac1ee640 2023-11-28T11:53:43.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.900+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c980682f0 msgr2=0x7f9c9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.900+0000 7f9cb1087700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c980682f0 0x7f9c9806a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0000c00 tx=0x7f9ca001e040 comp rx=0 tx=0).stop 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.900+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 msgr2=0x7f9cac1f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.900+0000 7f9cb1087700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 0x7f9cac1f0e50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9c940047d0 tx=0x7f9ca40123b0 comp rx=0 tx=0).stop 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 shutdown_connections 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac1ee640 0x7f9cac1f0e50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9c980682f0 0x7f9c9806a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:43.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c94003a30 0x7f9cac1eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:43.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 --2- 172.21.15.144:0/2015028591 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9cac1ee1d0 0x7f9cac1f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:43.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 >> 172.21.15.144:0/2015028591 conn(0x7f9cac06b380 msgr2=0x7f9cac0cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:43.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 shutdown_connections 2023-11-28T11:53:43.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:43.901+0000 7f9cb1087700 1 -- 172.21.15.144:0/2015028591 wait complete. 2023-11-28T11:53:43.903 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:53:43.914 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:53:44.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.040+0000 7fc64eab2700 1 Processor -- start 2023-11-28T11:53:44.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.040+0000 7fc64eab2700 1 -- start start 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64eab2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 0x7fc6480b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64eab2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 0x7fc6480b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64eab2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64806a270 con 0x7fc648065ef0 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64eab2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480b5040 con 0x7fc64815ac50 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64eab2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480b5320 con 0x7fc64815a880 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 0x7fc6480b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc647fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 0x7fc6480b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:44.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 0x7fc6480b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:50202/0 (socket says 172.21.15.144:50202) 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64d04f700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc648065ef0 0x7fc648071910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55322/0 (socket says 172.21.15.144:55322) 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc647fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 0x7fc6480b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:40394/0 (socket says 172.21.15.144:40394) 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 -- 172.21.15.144:0/900129696 learned_addr learned my addr 172.21.15.144:0/900129696 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 -- 172.21.15.144:0/900129696 >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 msgr2=0x7fc6480b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 --2- 172.21.15.144:0/900129696 >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 0x7fc6480b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 -- 172.21.15.144:0/900129696 >> v1:172.21.15.177:6789/0 conn(0x7fc648065ef0 legacy=0x7fc648071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc64c84e700 1 -- 172.21.15.144:0/900129696 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480d0020 con 0x7fc64815a880 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.041+0000 7fc647fff700 1 --2- 172.21.15.144:0/900129696 >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 0x7fc6480b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64c84e700 1 --2- 172.21.15.144:0/900129696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 0x7fc6480b4470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098f0 tx=0x7fc63c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93daa45d9f51bebb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc6477fe700 1 -- 172.21.15.144:0/900129696 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c011830 con 0x7fc64815a880 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc6477fe700 1 -- 172.21.15.144:0/900129696 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c011990 con 0x7fc64815a880 2023-11-28T11:53:44.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc6477fe700 1 -- 172.21.15.144:0/900129696 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c011d20 con 0x7fc64815a880 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 -- 172.21.15.144:0/900129696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 msgr2=0x7fc6480b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 --2- 172.21.15.144:0/900129696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 0x7fc6480b4470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0098f0 tx=0x7fc63c009c00 comp rx=0 tx=0).stop 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 -- 172.21.15.144:0/900129696 shutdown_connections 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 --2- 172.21.15.144:0/900129696 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815a880 0x7fc6480b4470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 --2- 172.21.15.144:0/900129696 >> v2:172.21.15.177:3300/0 conn(0x7fc64815ac50 0x7fc6480b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 -- 172.21.15.144:0/900129696 >> 172.21.15.144:0/900129696 conn(0x7fc6480b3080 msgr2=0x7fc6480cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 -- 172.21.15.144:0/900129696 shutdown_connections 2023-11-28T11:53:44.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.042+0000 7fc64eab2700 1 -- 172.21.15.144:0/900129696 wait complete. 2023-11-28T11:53:44.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.043+0000 7fc64eab2700 1 Processor -- start 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 -- start start 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc648065ef0 0x7fc648098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 0x7fc6480c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815ac50 0x7fc6480ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cec10 con 0x7fc64815a880 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cf0a0 con 0x7fc648065ef0 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.044+0000 7fc64eab2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481e11a0 con 0x7fc64815ac50 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 0x7fc6480c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 0x7fc6480c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38126/0 (socket says 172.21.15.144:38126) 2023-11-28T11:53:44.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 -- 172.21.15.144:0/2974353404 learned_addr learned my addr 172.21.15.144:0/2974353404 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc64c84e700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc648065ef0 0x7fc648098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc64d04f700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815ac50 0x7fc6480ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 -- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815ac50 msgr2=0x7fc6480ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815ac50 0x7fc6480ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 -- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc648065ef0 msgr2=0x7fc648098860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc648065ef0 0x7fc648098860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 -- 172.21.15.144:0/2974353404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6340094a0 con 0x7fc64815a880 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc647fff700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 0x7fc6480c9d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc63400e9b0 tx=0x7fc63400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634009700 con 0x7fc64815a880 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63400c850 con 0x7fc64815a880 2023-11-28T11:53:44.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c009550 con 0x7fc64815a880 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400cc00 con 0x7fc64815a880 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.045+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6480aec90 con 0x7fc64815a880 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.046+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6480d0300 con 0x7fc64815a880 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.046+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc634019030 con 0x7fc64815a880 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.047+0000 7fc6457fa700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc62c0682f0 0x7fc62c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.047+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fc634085ae0 con 0x7fc64815a880 2023-11-28T11:53:44.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.047+0000 7fc64c84e700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc62c0682f0 0x7fc62c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:44.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.047+0000 7fc64c84e700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc62c0682f0 0x7fc62c06a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000c00 tx=0x7fc63c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:44.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.051+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc634052400 con 0x7fc64815a880 2023-11-28T11:53:44.235 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.234+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc6480ae580 con 0x7fc64815a880 2023-11-28T11:53:44.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.897+0000 7fc6457fa700 1 -- 172.21.15.144:0/2974353404 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc634055a20 con 0x7fc64815a880 2023-11-28T11:53:44.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.901+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc62c0682f0 msgr2=0x7fc62c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:44.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.901+0000 7fc64eab2700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc62c0682f0 0x7fc62c06a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc63c000c00 tx=0x7fc63c008030 comp rx=0 tx=0).stop 2023-11-28T11:53:44.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.901+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 msgr2=0x7fc6480c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.901+0000 7fc64eab2700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 0x7fc6480c9d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc63400e9b0 tx=0x7fc63400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 shutdown_connections 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc62c0682f0 0x7fc62c06a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc64815ac50 0x7fc6480ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc648065ef0 0x7fc648098860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 --2- 172.21.15.144:0/2974353404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc64815a880 0x7fc6480c9d00 secure :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc63400e9b0 tx=0x7fc63400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 >> 172.21.15.144:0/2974353404 conn(0x7fc6480b3080 msgr2=0x7fc6480adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:44.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 shutdown_connections 2023-11-28T11:53:44.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:44.902+0000 7fc64eab2700 1 -- 172.21.15.144:0/2974353404 wait complete. 2023-11-28T11:53:44.904 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' removed 2023-11-28T11:53:44.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:53:45.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 Processor -- start 2023-11-28T11:53:45.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 -- start start 2023-11-28T11:53:45.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c7406e500 0x7f0c7415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c740ae9d0 0x7f0c7406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7406ee10 con 0x7f0c7415e540 2023-11-28T11:53:45.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7415bf10 con 0x7f0c7406e500 2023-11-28T11:53:45.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.053+0000 7f0c7a396700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c740aee40 con 0x7f0c740ae9d0 2023-11-28T11:53:45.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c7406e500 0x7f0c7415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c7406e500 0x7f0c7415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38132/0 (socket says 172.21.15.144:38132) 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 -- 172.21.15.144:0/809531554 learned_addr learned my addr 172.21.15.144:0/809531554 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c737fe700 1 --2- 172.21.15.144:0/809531554 >> v2:172.21.15.177:3300/0 conn(0x7f0c740ae9d0 0x7f0c7406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 -- 172.21.15.144:0/809531554 >> v2:172.21.15.177:3300/0 conn(0x7f0c740ae9d0 msgr2=0x7f0c7406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 --2- 172.21.15.144:0/809531554 >> v2:172.21.15.177:3300/0 conn(0x7f0c740ae9d0 0x7f0c7406e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 -- 172.21.15.144:0/809531554 >> v1:172.21.15.177:6789/0 conn(0x7f0c7415e540 legacy=0x7f0c7415e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 -- 172.21.15.144:0/809531554 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7406c930 con 0x7f0c7406e500 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c73fff700 1 --2- 172.21.15.144:0/809531554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c7415b9d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0c740719c0 tx=0x7f0c64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f946fdb1628172f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c72ffd700 1 -- 172.21.15.144:0/809531554 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400b870 con 0x7f0c7406e500 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c72ffd700 1 -- 172.21.15.144:0/809531554 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400b9d0 con 0x7f0c7406e500 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.054+0000 7f0c72ffd700 1 -- 172.21.15.144:0/809531554 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400bd60 con 0x7f0c7406e500 2023-11-28T11:53:45.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 -- 172.21.15.144:0/809531554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 msgr2=0x7f0c7415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 --2- 172.21.15.144:0/809531554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c7415b9d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0c740719c0 tx=0x7f0c64009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 -- 172.21.15.144:0/809531554 shutdown_connections 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 --2- 172.21.15.144:0/809531554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c7415b9d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 --2- 172.21.15.144:0/809531554 >> v2:172.21.15.177:3300/0 conn(0x7f0c740ae9d0 0x7f0c7406e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 -- 172.21.15.144:0/809531554 >> 172.21.15.144:0/809531554 conn(0x7f0c740b12b0 msgr2=0x7f0c740b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 -- 172.21.15.144:0/809531554 shutdown_connections 2023-11-28T11:53:45.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.055+0000 7f0c7a396700 1 -- 172.21.15.144:0/809531554 wait complete. 2023-11-28T11:53:45.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.056+0000 7f0c7a396700 1 Processor -- start 2023-11-28T11:53:45.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.056+0000 7f0c7a396700 1 -- start start 2023-11-28T11:53:45.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c7a396700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c740c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c7a396700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c740ae9d0 0x7f0c7414e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c7a396700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c7415e540 0x7f0c740c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c7a396700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74157dc0 con 0x7f0c7406e500 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c7a396700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74156ef0 con 0x7f0c740ae9d0 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c7a396700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74157140 con 0x7f0c7415e540 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c740c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c740c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38138/0 (socket says 172.21.15.144:38138) 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 -- 172.21.15.144:0/329819156 learned_addr learned my addr 172.21.15.144:0/329819156 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c737fe700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c740ae9d0 0x7f0c7414e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c78933700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c7415e540 0x7f0c740c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 -- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c7415e540 msgr2=0x7f0c740c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c7415e540 0x7f0c740c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 -- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c740ae9d0 msgr2=0x7f0c7414e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c740ae9d0 0x7f0c7414e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 -- 172.21.15.144:0/329819156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64009550 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c737fe700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c740ae9d0 0x7f0c7414e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c73fff700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c740c9300 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.057+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64004520 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.058+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6401e070 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.058+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c740664c0 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.058+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c640038a0 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.058+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7406cc10 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.059+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7406cef0 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.059+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c6400f050 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.059+0000 7f0c70ff9700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.060+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c64020070 con 0x7f0c7406e500 2023-11-28T11:53:45.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.060+0000 7f0c737fe700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.060+0000 7f0c737fe700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c741f4140 tx=0x7f0c68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:45.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.064+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c64053ad0 con 0x7f0c7406e500 2023-11-28T11:53:45.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.247+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0c74065fc0 con 0x7f0c7406e500 2023-11-28T11:53:45.252 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.251+0000 7f0c70ff9700 1 -- 172.21.15.144:0/329819156 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c640570f0 con 0x7f0c7406e500 2023-11-28T11:53:45.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 msgr2=0x7f0c6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c741f4140 tx=0x7f0c68008040 comp rx=0 tx=0).stop 2023-11-28T11:53:45.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 msgr2=0x7f0c740c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c740c9300 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c64004750 comp rx=0 tx=0).stop 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 shutdown_connections 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c7406e500 0x7f0c740c9300 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c7415e540 0x7f0c740c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 --2- 172.21.15.144:0/329819156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c740ae9d0 0x7f0c7414e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 >> 172.21.15.144:0/329819156 conn(0x7f0c740b12b0 msgr2=0x7f0c7415e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 shutdown_connections 2023-11-28T11:53:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.253+0000 7f0c7a396700 1 -- 172.21.15.144:0/329819156 wait complete. 2023-11-28T11:53:45.266 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:53:45.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.384+0000 7fd6617f7700 1 Processor -- start 2023-11-28T11:53:45.386 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.384+0000 7fd6617f7700 1 -- start start 2023-11-28T11:53:45.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd6617f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c15ca50 0x7fd65c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd6617f7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd65c0cce10 0x7fd65c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd6617f7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0cf5e0 con 0x7fd65c06b8c0 2023-11-28T11:53:45.388 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd6617f7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c0cd290 con 0x7fd65c0cce10 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd6617f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0cd470 con 0x7fd65c15ca50 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd65c0cce10 0x7fd65c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd65c0cce10 0x7fd65c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38150/0 (socket says 172.21.15.144:38150) 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 -- 172.21.15.144:0/1047194321 learned_addr learned my addr 172.21.15.144:0/1047194321 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65affd700 1 --2- 172.21.15.144:0/1047194321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c15ca50 0x7fd65c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 -- 172.21.15.144:0/1047194321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c15ca50 msgr2=0x7fd65c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 --2- 172.21.15.144:0/1047194321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c15ca50 0x7fd65c06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 -- 172.21.15.144:0/1047194321 >> v1:172.21.15.177:6789/0 conn(0x7fd65c06b8c0 legacy=0x7fd65c0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 -- 172.21.15.144:0/1047194321 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0cecb0 con 0x7fd65c0cce10 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd65b7fe700 1 --2- 172.21.15.144:0/1047194321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c0cce10 0x7fd65c0cfa70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd654009e80 tx=0x7fd65400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2314eb3fdaa93e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd659ffb700 1 -- 172.21.15.144:0/1047194321 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400e070 con 0x7fd65c0cce10 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd659ffb700 1 -- 172.21.15.144:0/1047194321 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654003a20 con 0x7fd65c0cce10 2023-11-28T11:53:45.389 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.385+0000 7fd659ffb700 1 -- 172.21.15.144:0/1047194321 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654003dd0 con 0x7fd65c0cce10 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 -- 172.21.15.144:0/1047194321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c0cce10 msgr2=0x7fd65c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 --2- 172.21.15.144:0/1047194321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c0cce10 0x7fd65c0cfa70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd654009e80 tx=0x7fd65400b330 comp rx=0 tx=0).stop 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 -- 172.21.15.144:0/1047194321 shutdown_connections 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 --2- 172.21.15.144:0/1047194321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c15ca50 0x7fd65c06b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 --2- 172.21.15.144:0/1047194321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c0cce10 0x7fd65c0cfa70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 -- 172.21.15.144:0/1047194321 >> 172.21.15.144:0/1047194321 conn(0x7fd65c09a230 msgr2=0x7fd65c08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 -- 172.21.15.144:0/1047194321 shutdown_connections 2023-11-28T11:53:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.386+0000 7fd6617f7700 1 -- 172.21.15.144:0/1047194321 wait complete. 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.387+0000 7fd6617f7700 1 Processor -- start 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.387+0000 7fd6617f7700 1 -- start start 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd6617f7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd65c06b8c0 0x7fd65c0b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd6617f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c0cce10 0x7fd65c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd6617f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 0x7fd65c06e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd6617f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c15d330 con 0x7fd65c15ca50 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd6617f7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0c4a60 con 0x7fd65c06b8c0 2023-11-28T11:53:45.393 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd6617f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c15f860 con 0x7fd65c0cce10 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 0x7fd65c06e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 0x7fd65c06e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38156/0 (socket says 172.21.15.144:38156) 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 -- 172.21.15.144:0/3884375746 learned_addr learned my addr 172.21.15.144:0/3884375746 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65affd700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd65c06b8c0 0x7fd65c0b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65a7fc700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c0cce10 0x7fd65c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 -- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c0cce10 msgr2=0x7fd65c06e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c0cce10 0x7fd65c06e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 -- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd65c06b8c0 msgr2=0x7fd65c0b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd65c06b8c0 0x7fd65c0b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.388+0000 7fd65b7fe700 1 -- 172.21.15.144:0/3884375746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd654009b30 con 0x7fd65c15ca50 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.389+0000 7fd65b7fe700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 0x7fd65c06e830 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd65c06d870 tx=0x7fd654004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.389+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654004440 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.389+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6540045c0 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.389+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c0cf270 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.389+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654004970 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.389+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c0cef90 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.390+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c15ce20 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.390+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd654022030 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.390+0000 7fd6437fe700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6480682f0 0x7fd64806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.391+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd654086dd0 con 0x7fd65c15ca50 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.391+0000 7fd65affd700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6480682f0 0x7fd64806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:45.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.391+0000 7fd65affd700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6480682f0 0x7fd64806a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0cfb00 tx=0x7fd64c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.395+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd654053930 con 0x7fd65c15ca50 2023-11-28T11:53:45.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.655+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fd65c0c4c20 con 0x7fd65c15ca50 2023-11-28T11:53:45.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.898+0000 7fd6437fe700 1 -- 172.21.15.144:0/3884375746 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v143) v1 ==== 149+0+0 (secure 0 0 0) 0x7fd654007310 con 0x7fd65c15ca50 2023-11-28T11:53:45.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6480682f0 msgr2=0x7fd64806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6480682f0 0x7fd64806a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0cfb00 tx=0x7fd64c008040 comp rx=0 tx=0).stop 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 msgr2=0x7fd65c06e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 0x7fd65c06e830 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd65c06d870 tx=0x7fd654004ad0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 shutdown_connections 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd65c15ca50 0x7fd65c06e830 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6480682f0 0x7fd64806a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd65c06b8c0 0x7fd65c0b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 --2- 172.21.15.144:0/3884375746 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd65c0cce10 0x7fd65c06e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.900+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 >> 172.21.15.144:0/3884375746 conn(0x7fd65c09a230 msgr2=0x7fd65c159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.901+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 shutdown_connections 2023-11-28T11:53:45.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:45.901+0000 7fd6617f7700 1 -- 172.21.15.144:0/3884375746 wait complete. 2023-11-28T11:53:45.914 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T11:53:45.915 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T11:53:45.915 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:53:45.915 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:53:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.034+0000 7fdb3eeed700 1 Processor -- start 2023-11-28T11:53:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.034+0000 7fdb3eeed700 1 -- start start 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3eeed700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb380ce040 0x7fdb3815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3eeed700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb380d0a00 0x7fdb3806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3eeed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380b2930 0x7fdb380b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3eeed700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb380b32f0 con 0x7fdb380ce040 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3eeed700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3806b420 con 0x7fdb380d0a00 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3eeed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3806b700 con 0x7fdb380b2930 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb380ce040 0x7fdb3815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb380ce040 0x7fdb3815df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38160/0 (socket says 172.21.15.144:38160) 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 -- 172.21.15.144:0/1841964283 learned_addr learned my addr 172.21.15.144:0/1841964283 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:46.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb37fff700 1 --2- 172.21.15.144:0/1841964283 >> v2:172.21.15.177:3300/0 conn(0x7fdb380d0a00 0x7fdb3806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3d48a700 1 --2- 172.21.15.144:0/1841964283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380b2930 0x7fdb380b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 -- 172.21.15.144:0/1841964283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380b2930 msgr2=0x7fdb380b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 --2- 172.21.15.144:0/1841964283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380b2930 0x7fdb380b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 -- 172.21.15.144:0/1841964283 >> v2:172.21.15.177:3300/0 conn(0x7fdb380d0a00 msgr2=0x7fdb3806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 --2- 172.21.15.144:0/1841964283 >> v2:172.21.15.177:3300/0 conn(0x7fdb380d0a00 0x7fdb3806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.035+0000 7fdb3cc89700 1 -- 172.21.15.144:0/1841964283 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3806b9e0 con 0x7fdb380ce040 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb37fff700 1 --2- 172.21.15.144:0/1841964283 >> v2:172.21.15.177:3300/0 conn(0x7fdb380d0a00 0x7fdb3806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3cc89700 1 --2- 172.21.15.144:0/1841964283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb3815df10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806ed50 tx=0x7fdb2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9f3c08a1c20fbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb377fe700 1 -- 172.21.15.144:0/1841964283 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c005e30 con 0x7fdb380ce040 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb377fe700 1 -- 172.21.15.144:0/1841964283 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c004030 con 0x7fdb380ce040 2023-11-28T11:53:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb377fe700 1 -- 172.21.15.144:0/1841964283 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0043e0 con 0x7fdb380ce040 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 -- 172.21.15.144:0/1841964283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 msgr2=0x7fdb3815df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/1841964283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb3815df10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806ed50 tx=0x7fdb2c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 -- 172.21.15.144:0/1841964283 shutdown_connections 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/1841964283 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380b2930 0x7fdb380b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/1841964283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb3815df10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/1841964283 >> v2:172.21.15.177:3300/0 conn(0x7fdb380d0a00 0x7fdb3806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 -- 172.21.15.144:0/1841964283 >> 172.21.15.144:0/1841964283 conn(0x7fdb3806aed0 msgr2=0x7fdb3806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 -- 172.21.15.144:0/1841964283 shutdown_connections 2023-11-28T11:53:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.036+0000 7fdb3eeed700 1 -- 172.21.15.144:0/1841964283 wait complete. 2023-11-28T11:53:46.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 Processor -- start 2023-11-28T11:53:46.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 -- start start 2023-11-28T11:53:46.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb380b2930 0x7fdb380c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb381f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380d0a00 0x7fdb380b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c69a0 con 0x7fdb380ce040 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c5170 con 0x7fdb380b2930 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.038+0000 7fdb3eeed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38153410 con 0x7fdb380d0a00 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb381f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb381f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38168/0 (socket says 172.21.15.144:38168) 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 -- 172.21.15.144:0/2087150410 learned_addr learned my addr 172.21.15.144:0/2087150410 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb3d48a700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380d0a00 0x7fdb380b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb3cc89700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb380b2930 0x7fdb380c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 -- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380d0a00 msgr2=0x7fdb380b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380d0a00 0x7fdb380b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 -- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb380b2930 msgr2=0x7fdb380c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb380b2930 0x7fdb380c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 -- 172.21.15.144:0/2087150410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c009500 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb37fff700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb381f1870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c750 tx=0x7fdb2400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24011800 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24011960 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3806bfa0 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24011cd0 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.039+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3806bcc0 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.040+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3806fd90 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.040+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb24020070 con 0x7fdb380ce040 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.041+0000 7fdb357fa700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb1c0682f0 0x7fdb1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.041+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdb24085590 con 0x7fdb380ce040 2023-11-28T11:53:46.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.041+0000 7fdb3cc89700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb1c0682f0 0x7fdb1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:46.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.041+0000 7fdb3cc89700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb1c0682f0 0x7fdb1c06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806ed50 tx=0x7fdb2c008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:46.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.045+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb240520f0 con 0x7fdb380ce040 2023-11-28T11:53:46.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.299+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fdb38068690 con 0x7fdb380ce040 2023-11-28T11:53:46.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.907+0000 7fdb357fa700 1 -- 172.21.15.144:0/2087150410 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v144) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdb24055710 con 0x7fdb380ce040 2023-11-28T11:53:46.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb1c0682f0 msgr2=0x7fdb1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:46.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb1c0682f0 0x7fdb1c06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806ed50 tx=0x7fdb2c008070 comp rx=0 tx=0).stop 2023-11-28T11:53:46.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 msgr2=0x7fdb381f1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:46.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb381f1870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400c750 tx=0x7fdb2400ca60 comp rx=0 tx=0).stop 2023-11-28T11:53:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 shutdown_connections 2023-11-28T11:53:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fdb1c0682f0 0x7fdb1c06a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb380d0a00 0x7fdb380b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb380b2930 0x7fdb380c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 --2- 172.21.15.144:0/2087150410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb380ce040 0x7fdb381f1870 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:46.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 >> 172.21.15.144:0/2087150410 conn(0x7fdb3806aed0 msgr2=0x7fdb3815e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:46.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 shutdown_connections 2023-11-28T11:53:46.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:46.910+0000 7fdb3eeed700 1 -- 172.21.15.144:0/2087150410 wait complete. 2023-11-28T11:53:46.928 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:53:47.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.056+0000 7f9493251700 1 Processor -- start 2023-11-28T11:53:47.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 -- start start 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f948c071040 0x7f948c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f948c06dee0 0x7f948c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c071950 con 0x7f948c071040 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c15d060 con 0x7f948c06dee0 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.057+0000 7f9493251700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c15d340 con 0x7f948c06cdc0 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f948c071040 0x7f948c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f948c071040 0x7f948c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38174/0 (socket says 172.21.15.144:38174) 2023-11-28T11:53:47.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 -- 172.21.15.144:0/3081762503 learned_addr learned my addr 172.21.15.144:0/3081762503 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f94917ee700 1 --2- 172.21.15.144:0/3081762503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f948bfff700 1 --2- 172.21.15.144:0/3081762503 >> v2:172.21.15.177:3300/0 conn(0x7f948c06dee0 0x7f948c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 -- 172.21.15.144:0/3081762503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 msgr2=0x7f948c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 --2- 172.21.15.144:0/3081762503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c06d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 -- 172.21.15.144:0/3081762503 >> v2:172.21.15.177:3300/0 conn(0x7f948c06dee0 msgr2=0x7f948c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 --2- 172.21.15.144:0/3081762503 >> v2:172.21.15.177:3300/0 conn(0x7f948c06dee0 0x7f948c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 -- 172.21.15.144:0/3081762503 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c15d620 con 0x7f948c071040 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f9490fed700 1 --2- 172.21.15.144:0/3081762503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c071410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f948c06fb00 tx=0x7f9480009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67534eae7f59bc2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f948b7fe700 1 -- 172.21.15.144:0/3081762503 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000b870 con 0x7f948c071040 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f948b7fe700 1 -- 172.21.15.144:0/3081762503 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948000b9d0 con 0x7f948c071040 2023-11-28T11:53:47.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.058+0000 7f948b7fe700 1 -- 172.21.15.144:0/3081762503 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000bd60 con 0x7f948c071040 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 -- 172.21.15.144:0/3081762503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 msgr2=0x7f948c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 --2- 172.21.15.144:0/3081762503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c071410 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f948c06fb00 tx=0x7f9480009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 -- 172.21.15.144:0/3081762503 shutdown_connections 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 --2- 172.21.15.144:0/3081762503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 --2- 172.21.15.144:0/3081762503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c071410 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 --2- 172.21.15.144:0/3081762503 >> v2:172.21.15.177:3300/0 conn(0x7f948c06dee0 0x7f948c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 -- 172.21.15.144:0/3081762503 >> 172.21.15.144:0/3081762503 conn(0x7f948c0d0000 msgr2=0x7f948c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 -- 172.21.15.144:0/3081762503 shutdown_connections 2023-11-28T11:53:47.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.059+0000 7f9493251700 1 -- 172.21.15.144:0/3081762503 wait complete. 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.060+0000 7f9493251700 1 Processor -- start 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.060+0000 7f9493251700 1 -- start start 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9493251700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9493251700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f948c06dee0 0x7f948c152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9493251700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c0b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9493251700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0c3aa0 con 0x7f948c071040 2023-11-28T11:53:47.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9493251700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0c3d80 con 0x7f948c06dee0 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9493251700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0c4060 con 0x7f948c06cdc0 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c0b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c0b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38184/0 (socket says 172.21.15.144:38184) 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 -- 172.21.15.144:0/2116271614 learned_addr learned my addr 172.21.15.144:0/2116271614 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f9490fed700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f948bfff700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f948c06dee0 0x7f948c152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 -- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 msgr2=0x7f948c154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 -- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f948c06dee0 msgr2=0x7f948c152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f948c06dee0 0x7f948c152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f94917ee700 1 -- 172.21.15.144:0/2116271614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009550 con 0x7f948c071040 2023-11-28T11:53:47.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.061+0000 7f948bfff700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f948c06dee0 0x7f948c152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.062+0000 7f94917ee700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c0b4780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9484007d20 tx=0x7f948400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.062+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400ede0 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.062+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9484014420 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.062+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c06e870 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.062+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94840147d0 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.062+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c15d900 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.063+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c15ccd0 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.063+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9484012050 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.064+0000 7f94897fa700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94700682f0 0x7f947006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.064+0000 7f9490fed700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94700682f0 0x7f947006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.064+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9484085f50 con 0x7f948c071040 2023-11-28T11:53:47.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.064+0000 7f9490fed700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94700682f0 0x7f947006a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9480009840 tx=0x7f9480008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:47.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.068+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9484052a50 con 0x7f948c071040 2023-11-28T11:53:47.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.252+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f948c0c6d70 con 0x7f948c071040 2023-11-28T11:53:47.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.258+0000 7f94897fa700 1 -- 172.21.15.144:0/2116271614 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9484056070 con 0x7f948c071040 2023-11-28T11:53:47.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94700682f0 msgr2=0x7f947006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94700682f0 0x7f947006a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9480009840 tx=0x7f9480008030 comp rx=0 tx=0).stop 2023-11-28T11:53:47.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 msgr2=0x7f948c0b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c0b4780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9484007d20 tx=0x7f948400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 shutdown_connections 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f948c071040 0x7f948c0b4780 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94700682f0 0x7f947006a7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f948c06cdc0 0x7f948c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 --2- 172.21.15.144:0/2116271614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f948c06dee0 0x7f948c152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 >> 172.21.15.144:0/2116271614 conn(0x7f948c0d0000 msgr2=0x7f948c15e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 shutdown_connections 2023-11-28T11:53:47.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.260+0000 7f9493251700 1 -- 172.21.15.144:0/2116271614 wait complete. 2023-11-28T11:53:47.274 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:53:47.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:53:47.319 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:53:47.362 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:53:47.362 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:53:47.362 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:47.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:53:47.379 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:53:47.379 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:47.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:53:47.422 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:53:47.422 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:47.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:53:47.440 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:53:47.440 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:53:47.481 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:53:47.481 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:53:47.481 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:47.601 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.599+0000 7f2c7a9c5700 1 Processor -- start 2023-11-28T11:53:47.601 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.599+0000 7f2c7a9c5700 1 -- start start 2023-11-28T11:53:47.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c7a9c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160740 0x7f2c740cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c7a9c5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c74160b10 0x7f2c740cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c7a9c5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c7406b060 0x7f2c7406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c7a9c5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c740ca720 con 0x7f2c74160b10 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c7a9c5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c740cc030 con 0x7f2c7406b060 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c7a9c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740c9e10 con 0x7f2c74160740 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c74160b10 0x7f2c740cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c74160b10 0x7f2c740cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38190/0 (socket says 172.21.15.144:38190) 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 -- 172.21.15.144:0/4075248969 learned_addr learned my addr 172.21.15.144:0/4075248969 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c78f62700 1 --2- 172.21.15.144:0/4075248969 >> v2:172.21.15.177:3300/0 conn(0x7f2c7406b060 0x7f2c7406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c73fff700 1 --2- 172.21.15.144:0/4075248969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160740 0x7f2c740cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 -- 172.21.15.144:0/4075248969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160740 msgr2=0x7f2c740cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:47.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4075248969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160740 0x7f2c740cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 -- 172.21.15.144:0/4075248969 >> v2:172.21.15.177:3300/0 conn(0x7f2c7406b060 msgr2=0x7f2c7406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4075248969 >> v2:172.21.15.177:3300/0 conn(0x7f2c7406b060 0x7f2c7406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c737fe700 1 -- 172.21.15.144:0/4075248969 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7406efc0 con 0x7f2c74160b10 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.600+0000 7f2c78f62700 1 --2- 172.21.15.144:0/4075248969 >> v2:172.21.15.177:3300/0 conn(0x7f2c7406b060 0x7f2c7406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4075248969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c74160b10 0x7f2c740cd390 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2c7415e2e0 tx=0x7f2c64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10586eeffa91773 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c72ffd700 1 -- 172.21.15.144:0/4075248969 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400b870 con 0x7f2c74160b10 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c72ffd700 1 -- 172.21.15.144:0/4075248969 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400b9d0 con 0x7f2c74160b10 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c72ffd700 1 -- 172.21.15.144:0/4075248969 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400bd60 con 0x7f2c74160b10 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4075248969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c74160b10 msgr2=0x7f2c740cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4075248969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c74160b10 0x7f2c740cd390 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2c7415e2e0 tx=0x7f2c64009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4075248969 shutdown_connections 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4075248969 >> v2:172.21.15.177:3300/0 conn(0x7f2c7406b060 0x7f2c7406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4075248969 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160740 0x7f2c740cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4075248969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c74160b10 0x7f2c740cd390 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4075248969 >> 172.21.15.144:0/4075248969 conn(0x7f2c740b12b0 msgr2=0x7f2c740b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:47.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4075248969 shutdown_connections 2023-11-28T11:53:47.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.601+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4075248969 wait complete. 2023-11-28T11:53:47.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 Processor -- start 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 -- start start 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 0x7f2c741505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c74160740 0x7f2c740c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160b10 0x7f2c7415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c741604a0 con 0x7f2c7406b060 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7415f400 con 0x7f2c74160740 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.603+0000 7f2c7a9c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7415f6e0 con 0x7f2c74160b10 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 0x7f2c741505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 0x7f2c741505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38194/0 (socket says 172.21.15.144:38194) 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 -- 172.21.15.144:0/4106293999 learned_addr learned my addr 172.21.15.144:0/4106293999 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 -- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160b10 msgr2=0x7f2c7415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160b10 0x7f2c7415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 -- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c74160740 msgr2=0x7f2c740c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c74160740 0x7f2c740c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c74160740 0x7f2c740c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 -- 172.21.15.144:0/4106293999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009550 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c74160740 0x7f2c740c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c73fff700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 0x7f2c741505b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00e810 tx=0x7f2c5c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c013610 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c013770 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c740ca3b0 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c013b20 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.604+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c740ca0d0 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.605+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c740cd8d0 con 0x7f2c7406b060 2023-11-28T11:53:47.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.606+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c5c023070 con 0x7f2c7406b060 2023-11-28T11:53:47.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.606+0000 7f2c70ff9700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c540682f0 0x7f2c5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:47.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.606+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c540682f0 0x7f2c5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:47.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.606+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2c5c0856c0 con 0x7f2c7406b060 2023-11-28T11:53:47.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.606+0000 7f2c737fe700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c540682f0 0x7f2c5406a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2c741f5cb0 tx=0x7f2c64008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:47.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.610+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c5c0521a0 con 0x7f2c7406b060 2023-11-28T11:53:47.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.835+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c741602f0 con 0x7f2c7406b060 2023-11-28T11:53:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.836+0000 7f2c70ff9700 1 -- 172.21.15.144:0/4106293999 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f2c5c021df0 con 0x7f2c7406b060 2023-11-28T11:53:47.838 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:47.838 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4943,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1022806032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1022806032},{"type":"v1","addr":"172.21.15.144:6837","nonce":1022806032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5210,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/599351921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":599351921},{"type":"v1","addr":"172.21.15.177:6834","nonce":599351921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5233,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/1090442377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1090442377},{"type":"v1","addr":"172.21.15.177:6835","nonce":1090442377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3317568420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3317568420},{"type":"v1","addr":"172.21.15.144:6836","nonce":3317568420}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c540682f0 msgr2=0x7f2c5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c540682f0 0x7f2c5406a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2c741f5cb0 tx=0x7f2c64008030 comp rx=0 tx=0).stop 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 msgr2=0x7f2c741505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 0x7f2c741505b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00e810 tx=0x7f2c5c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 shutdown_connections 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c7406b060 0x7f2c741505b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c540682f0 0x7f2c5406a7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c74160b10 0x7f2c7415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 --2- 172.21.15.144:0/4106293999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c74160740 0x7f2c740c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 >> 172.21.15.144:0/4106293999 conn(0x7f2c740b12b0 msgr2=0x7f2c740cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 shutdown_connections 2023-11-28T11:53:47.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:47.838+0000 7f2c7a9c5700 1 -- 172.21.15.144:0/4106293999 wait complete. 2023-11-28T11:53:47.841 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 26 2023-11-28T11:53:47.852 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:53:47.852 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:53:47.852 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:53:47.853 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:53:47.853 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:53:47.853 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:53:47.853 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:53:47.853 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:53:47.853 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:53:47.853 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:53:47.854 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:53:47.854 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:53:47.854 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:53:47.854 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:53:47.854 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:53:47.854 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:53:47.864 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:53:47.864 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:53:47.865 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:53:47.865 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:53:47.866 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:53:47.866 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:53:47.867 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:53:47.867 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:53:47.906 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:53:47.906 DEBUG:teuthology.parallel:result is None 2023-11-28T11:53:47.907 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:53:47.908 DEBUG:teuthology.parallel:result is None 2023-11-28T11:53:47.909 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:53:47.909 DEBUG:teuthology.parallel:result is None 2023-11-28T11:53:47.909 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:53:47.910 DEBUG:teuthology.parallel:result is None 2023-11-28T11:53:47.910 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:53:47.985 INFO:tasks.ceph.mds.c.smithi144.stdout:starting mds.c at 2023-11-28T11:53:47.985 INFO:tasks.ceph.mds.a.smithi144.stdout:starting mds.a at 2023-11-28T11:53:47.999 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-11-28T11:53:47.999 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-11-28T11:53:48.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.040+0000 7f644271c700 1 Processor -- start 2023-11-28T11:53:48.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.040+0000 7f644271c700 1 -- start start 2023-11-28T11:53:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f644271c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f643c15c9b0 0x7f643c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f644271c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 0x7f643c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f644271c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c0d1950 con 0x7f643c161340 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f644271c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0b5800 con 0x7f643c15c9b0 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f644271c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0ce140 con 0x7f643c0cd7c0 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f6440cb9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 0x7f643c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f6440cb9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 0x7f643c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:50312/0 (socket says 172.21.15.144:50312) 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f643bfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f643c15c9b0 0x7f643c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f6440cb9700 1 -- 172.21.15.144:0/1009790976 learned_addr learned my addr 172.21.15.144:0/1009790976 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f643b7fe700 1 --1- 172.21.15.144:0/1009790976 >> v1:172.21.15.177:6789/0 conn(0x7f643c161340 0x7f643c0d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55428/0 (socket says 172.21.15.144:55428) 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f6440cb9700 1 -- 172.21.15.144:0/1009790976 >> v2:172.21.15.177:3300/0 conn(0x7f643c15c9b0 msgr2=0x7f643c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.041+0000 7f6440cb9700 1 --2- 172.21.15.144:0/1009790976 >> v2:172.21.15.177:3300/0 conn(0x7f643c15c9b0 0x7f643c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:48.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f6440cb9700 1 -- 172.21.15.144:0/1009790976 >> v1:172.21.15.177:6789/0 conn(0x7f643c161340 legacy=0x7f643c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:48.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f6440cb9700 1 -- 172.21.15.144:0/1009790976 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c0d24f0 con 0x7f643c0cd7c0 2023-11-28T11:53:48.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f643bfff700 1 --2- 172.21.15.144:0/1009790976 >> v2:172.21.15.177:3300/0 conn(0x7f643c15c9b0 0x7f643c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:48.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f6440cb9700 1 --2- 172.21.15.144:0/1009790976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 0x7f643c0cdc00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e50 tx=0x7f643400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd2987c58ea985d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:48.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f643affd700 1 -- 172.21.15.144:0/1009790976 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e070 con 0x7f643c0cd7c0 2023-11-28T11:53:48.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f643affd700 1 -- 172.21.15.144:0/1009790976 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400b910 con 0x7f643c0cd7c0 2023-11-28T11:53:48.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.042+0000 7f643affd700 1 -- 172.21.15.144:0/1009790976 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400bcc0 con 0x7f643c0cd7c0 2023-11-28T11:53:48.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 -- 172.21.15.144:0/1009790976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 msgr2=0x7f643c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:48.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 --2- 172.21.15.144:0/1009790976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 0x7f643c0cdc00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e50 tx=0x7f643400b330 comp rx=0 tx=0).stop 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 -- 172.21.15.144:0/1009790976 shutdown_connections 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 --2- 172.21.15.144:0/1009790976 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c0cd7c0 0x7f643c0cdc00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 --2- 172.21.15.144:0/1009790976 >> v2:172.21.15.177:3300/0 conn(0x7f643c15c9b0 0x7f643c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 -- 172.21.15.144:0/1009790976 >> 172.21.15.144:0/1009790976 conn(0x7f643c072600 msgr2=0x7f643c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 -- 172.21.15.144:0/1009790976 shutdown_connections 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.043+0000 7f644271c700 1 -- 172.21.15.144:0/1009790976 wait complete. 2023-11-28T11:53:48.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.045+0000 7f644271c700 1 Processor -- start 2023-11-28T11:53:48.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.045+0000 7f644271c700 1 -- start start 2023-11-28T11:53:48.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f644271c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f643c0cd7c0 0x7f643c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:48.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f644271c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c15c9b0 0x7f643c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f644271c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 0x7f643c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f644271c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c154990 con 0x7f643c161340 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f644271c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0cae20 con 0x7f643c0cd7c0 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f644271c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0cb100 con 0x7f643c15c9b0 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 0x7f643c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 0x7f643c158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38236/0 (socket says 172.21.15.144:38236) 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 -- 172.21.15.144:0/1249241918 learned_addr learned my addr 172.21.15.144:0/1249241918 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f643b7fe700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c15c9b0 0x7f643c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f643bfff700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f643c0cd7c0 0x7f643c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 -- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c15c9b0 msgr2=0x7f643c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c15c9b0 0x7f643c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:48.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 -- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f643c0cd7c0 msgr2=0x7f643c14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f643c0cd7c0 0x7f643c14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 -- 172.21.15.144:0/1249241918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300094a0 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.046+0000 7f6440cb9700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 0x7f643c158db0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f643c067550 tx=0x7f6434004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.047+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434004800 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.047+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434004960 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434009b30 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.047+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643401b670 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c0d1300 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c0b5520 con 0x7f643c161340 2023-11-28T11:53:48.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.048+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f643402b070 con 0x7f643c161340 2023-11-28T11:53:48.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.048+0000 7f6438ff9700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f64280682f0 0x7f642806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:48.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.048+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6434087160 con 0x7f643c161340 2023-11-28T11:53:48.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.048+0000 7f643bfff700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f64280682f0 0x7f642806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:48.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.049+0000 7f643bfff700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f64280682f0 0x7f642806a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6430009cd0 tx=0x7f6430009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:48.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.052+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6434053bf0 con 0x7f643c161340 2023-11-28T11:53:48.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:48.249+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f643c0d15e0 con 0x7f643c161340 2023-11-28T11:53:49.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.045+0000 7f6438ff9700 1 -- 172.21.15.144:0/1249241918 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6434057210 con 0x7f643c161340 2023-11-28T11:53:49.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f64280682f0 msgr2=0x7f642806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f64280682f0 0x7f642806a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6430009cd0 tx=0x7f6430009210 comp rx=0 tx=0).stop 2023-11-28T11:53:49.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 msgr2=0x7f643c158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 0x7f643c158db0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f643c067550 tx=0x7f6434004ad0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 shutdown_connections 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f643c161340 0x7f643c158db0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f64280682f0 0x7f642806a7a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f643c0cd7c0 0x7f643c14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 --2- 172.21.15.144:0/1249241918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f643c15c9b0 0x7f643c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.047+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 >> 172.21.15.144:0/1249241918 conn(0x7f643c072600 msgr2=0x7f643c0b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.048+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 shutdown_connections 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.048+0000 7f644271c700 1 -- 172.21.15.144:0/1249241918 wait complete. 2023-11-28T11:53:49.050 INFO:teuthology.orchestra.run.smithi144.stderr: removed all blocklist entries 2023-11-28T11:53:49.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:53:49.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 Processor -- start 2023-11-28T11:53:49.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 -- start start 2023-11-28T11:53:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0b600d01b0 0x7f0b600d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 0x7f0b600d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b600b0230 con 0x7f0b600b20b0 2023-11-28T11:53:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6015cec0 con 0x7f0b600d01b0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.184+0000 7f0b65c5b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600b4570 con 0x7f0b600b16d0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5effd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0b600b20b0 0x7f0b600b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54956/0 (socket says 172.21.15.144:54956) 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5effd700 1 -- 172.21.15.144:0/1442989264 learned_addr learned my addr 172.21.15.144:0/1442989264 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634052601 0 0) 0x7f0b600b0230 con 0x7f0b600b20b0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/1442989264 >> v2:172.21.15.177:3300/0 conn(0x7f0b600d01b0 0x7f0b600d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b4c0035b0 con 0x7f0b600b20b0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5ffff700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 0x7f0b600d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942304549 0 0) 0x7f0b4c0035b0 con 0x7f0b600b20b0 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 msgr2=0x7f0b600d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 0x7f0b600d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 >> v2:172.21.15.177:3300/0 conn(0x7f0b600d01b0 msgr2=0x7f0b600d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 --2- 172.21.15.144:0/1442989264 >> v2:172.21.15.177:3300/0 conn(0x7f0b600d01b0 0x7f0b600d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600b4850 con 0x7f0b600b20b0 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f0b50001670 con 0x7f0b600b20b0 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 >> v1:172.21.15.144:6789/0 conn(0x7f0b600b20b0 legacy=0x7f0b600b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b4c003a00 0x7f0b4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c004430 0x7f0b4c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 0x7f0b4c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6015cec0 con 0x7f0b4c003a00 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600b4570 con 0x7f0b4c004430 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c00ba10 con 0x7f0b600b16d0 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b4c003a00 0x7f0b4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5effd700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c004430 0x7f0b4c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.185+0000 7f0b5ffff700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 0x7f0b4c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5f7fe700 1 -- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 msgr2=0x7f0b4c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b600b16d0 0x7f0b4c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5f7fe700 1 -- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c004430 msgr2=0x7f0b4c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c004430 0x7f0b4c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5f7fe700 1 -- 172.21.15.144:0/1442989264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0035b0 con 0x7f0b4c003a00 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b4c003a00 0x7f0b4c003e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009b60 tx=0x7f0b4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c50de969147fb59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800e6c0 con 0x7f0b4c003a00 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4800e820 con 0x7f0b4c003a00 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5e7fc700 1 -- 172.21.15.144:0/1442989264 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800eb90 con 0x7f0b4c003a00 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b65c5b700 1 -- 172.21.15.144:0/1442989264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b4c003a00 msgr2=0x7f0b4c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/1442989264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b4c003a00 0x7f0b4c003e10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009b60 tx=0x7f0b4c005880 comp rx=0 tx=0).stop 2023-11-28T11:53:49.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.186+0000 7f0b5ffff700 1 -- 172.21.15.144:0/1442989264 reap_dead start 2023-11-28T11:53:49.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.187+0000 7f0b65c5b700 1 -- 172.21.15.144:0/1442989264 shutdown_connections 2023-11-28T11:53:49.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.187+0000 7f0b65c5b700 1 -- 172.21.15.144:0/1442989264 >> 172.21.15.144:0/1442989264 conn(0x7f0b6009c3f0 msgr2=0x7f0b600d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:49.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.187+0000 7f0b65c5b700 1 -- 172.21.15.144:0/1442989264 shutdown_connections 2023-11-28T11:53:49.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.187+0000 7f0b65c5b700 1 -- 172.21.15.144:0/1442989264 wait complete. 2023-11-28T11:53:49.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.188+0000 7f0b65c5b700 1 Processor -- start 2023-11-28T11:53:49.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.188+0000 7f0b65c5b700 1 -- start start 2023-11-28T11:53:49.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b65c5b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c003a00 0x7f0b600cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b65c5b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 0x7f0b60151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b65c5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b6015f050 0x7f0b6015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b65c5b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6015f9e0 con 0x7f0b60152460 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b65c5b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600cab30 con 0x7f0b4c003a00 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b65c5b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b600cad80 con 0x7f0b6015f050 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b5effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 0x7f0b60151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b5effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 0x7f0b60151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38254/0 (socket says 172.21.15.144:38254) 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b5effd700 1 -- 172.21.15.144:0/461841980 learned_addr learned my addr 172.21.15.144:0/461841980 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c003a00 0x7f0b600cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.189+0000 7f0b5ffff700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b6015f050 0x7f0b6015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b5effd700 1 -- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b6015f050 msgr2=0x7f0b6015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b5effd700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b6015f050 0x7f0b6015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b5effd700 1 -- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c003a00 msgr2=0x7f0b600cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b5effd700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c003a00 0x7f0b600cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b5effd700 1 -- 172.21.15.144:0/461841980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600afec0 con 0x7f0b60152460 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b5effd700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 0x7f0b60151bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000fb0 tx=0x7f0b500075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50011580 con 0x7f0b60152460 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b500116e0 con 0x7f0b60152460 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b600afbe0 con 0x7f0b60152460 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50011a90 con 0x7f0b60152460 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.190+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b600b2600 con 0x7f0b60152460 2023-11-28T11:53:49.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.191+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b60068690 con 0x7f0b60152460 2023-11-28T11:53:49.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.191+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b50011bf0 con 0x7f0b60152460 2023-11-28T11:53:49.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.192+0000 7f0b57fff700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0b400682f0 0x7f0b4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.192+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0b50085950 con 0x7f0b60152460 2023-11-28T11:53:49.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.192+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0b400682f0 0x7f0b4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.192+0000 7f0b5f7fe700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0b400682f0 0x7f0b4006a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0b480090b0 tx=0x7f0b4801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.196+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0b50052500 con 0x7f0b60152460 2023-11-28T11:53:49.451 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.450+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0b600662f0 con 0x7f0b60152460 2023-11-28T11:53:49.452 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.450+0000 7f0b57fff700 1 -- 172.21.15.144:0/461841980 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f0b50055b20 con 0x7f0b60152460 2023-11-28T11:53:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:49.452 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "auth_dump": [ 2023-11-28T11:53:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.452 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.a", 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlzuwFOBAAtN3Mhkt5bwjiPFumLWf5LQ==", 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.b", 2023-11-28T11:53:49.453 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlmkasCBAAaYU+0cOXvyDnawSPLqNQBg==", 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.c", 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlyo2xAxAARq98KYTF0ViGGgRsgbiHJg==", 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.454 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mds.d", 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVltR3sDxAAkRdNJ3ar7Vh/OwKimeVIGQ==", 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.455 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.0", 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAE1GVlRYZuNBAAkRLYVWzVGa/wVN7KApc9pA==", 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.456 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.1", 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAH1GVlzE4mNBAA8fJmDI1X35Kb7ffo7fKFGQ==", 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.457 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.2", 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAK1GVl0BB5OBAAJXjaD/H42vlurqW5ReH8QQ==", 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.3", 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAN1GVllXIyOBAAHYoY70u/C1a0B/IZhBZ3kw==", 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.458 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.4", 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAU1GVlWMw2ARAAQDzOPIY+tj6nPKoQ5d1Opw==", 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.459 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.5", 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAX1GVlwfFcABAAzJkyTNckY6Ua/4KuHrTn2A==", 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.460 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.6", 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAa1GVldpcCABAAY7IyYyuR3TMLzueAnd70vA==", 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "osd.7", 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAc1GVlnehWOxAA+UrMHeuJP4Btd7AlMXMyYA==", 2023-11-28T11:53:49.461 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow profile osd", 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile osd", 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.0", 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVl4NRbFBAA6tNlYbw6YiCQ4X7ewAOxAw==", 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.462 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow r", 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.1", 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlfaJnHRAA4AdUVgWb3JW1D50Vj+quDQ==", 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:53:49.463 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.2", 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlxfcVGhAAEX5yipIE6u3NvW4h8fLhjw==", 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.464 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.3", 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAB1GVlF0KoIBAAewnQNQ6UxhSPTRCTU/mgtQ==", 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow", 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow r", 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow rw", 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow rwx" 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.465 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "client.admin", 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlkOuRGxAAVRU3tb5dFIW29oanM4hW6g==", 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": "allow *", 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow *", 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mgr.x", 2023-11-28T11:53:49.466 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVl+4ibNBAAmnsKW1HqPNEAHCtefjHz4A==", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile mgr", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "entity": "mgr.y", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "AQAA1GVlU3zUMBAAYSJLQSW0CAYPWfcdqxqHbA==", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": { 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": "allow *", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": "allow profile mgr", 2023-11-28T11:53:49.467 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": "allow *" 2023-11-28T11:53:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:49.468 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2023-11-28T11:53:49.468 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0b400682f0 msgr2=0x7f0b4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0b400682f0 0x7f0b4006a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0b480090b0 tx=0x7f0b4801f040 comp rx=0 tx=0).stop 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 msgr2=0x7f0b60151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 0x7f0b60151bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000fb0 tx=0x7f0b500075f0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 shutdown_connections 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0b400682f0 0x7f0b4006a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b6015f050 0x7f0b6015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c003a00 0x7f0b600cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 --2- 172.21.15.144:0/461841980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b60152460 0x7f0b60151bf0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 >> 172.21.15.144:0/461841980 conn(0x7f0b6009c3f0 msgr2=0x7f0b6006ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 shutdown_connections 2023-11-28T11:53:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.452+0000 7f0b65c5b700 1 -- 172.21.15.144:0/461841980 wait complete. 2023-11-28T11:53:49.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:49.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.589+0000 7fe2b744c700 1 Processor -- start 2023-11-28T11:53:49.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.589+0000 7fe2b744c700 1 -- start start 2023-11-28T11:53:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b744c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2b0154e70 0x7fe2b00cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b744c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b00cf360 0x7fe2b0071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b744c700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00719e0 con 0x7fe2b0071580 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b744c700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b0155240 con 0x7fe2b0154e70 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b744c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b015b100 con 0x7fe2b00cf360 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b51e8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2b0154e70 0x7fe2b00cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b59e9700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe2b0071580 0x7fe2b015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54972/0 (socket says 172.21.15.144:54972) 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.590+0000 7fe2b59e9700 1 -- 172.21.15.144:0/4057527281 learned_addr learned my addr 172.21.15.144:0/4057527281 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b49e7700 1 --2- 172.21.15.144:0/4057527281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b00cf360 0x7fe2b0071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b51e8700 1 -- 172.21.15.144:0/4057527281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b00cf360 msgr2=0x7fe2b0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b51e8700 1 --2- 172.21.15.144:0/4057527281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b00cf360 0x7fe2b0071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b51e8700 1 -- 172.21.15.144:0/4057527281 >> v1:172.21.15.144:6789/0 conn(0x7fe2b0071580 legacy=0x7fe2b015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b51e8700 1 -- 172.21.15.144:0/4057527281 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b015b3e0 con 0x7fe2b0154e70 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b51e8700 1 --2- 172.21.15.144:0/4057527281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0154e70 0x7fe2b00cdbd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009840 tx=0x7fe2a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa23f90036d50e5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2a7fff700 1 -- 172.21.15.144:0/4057527281 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0011830 con 0x7fe2b0154e70 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2a7fff700 1 -- 172.21.15.144:0/4057527281 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a0011990 con 0x7fe2b0154e70 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2a7fff700 1 -- 172.21.15.144:0/4057527281 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0011d20 con 0x7fe2b0154e70 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b744c700 1 -- 172.21.15.144:0/4057527281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0154e70 msgr2=0x7fe2b00cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.591+0000 7fe2b744c700 1 --2- 172.21.15.144:0/4057527281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0154e70 0x7fe2b00cdbd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009840 tx=0x7fe2a0009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.592+0000 7fe2b744c700 1 -- 172.21.15.144:0/4057527281 shutdown_connections 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.592+0000 7fe2b744c700 1 --2- 172.21.15.144:0/4057527281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0154e70 0x7fe2b00cdbd0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.592+0000 7fe2b744c700 1 --2- 172.21.15.144:0/4057527281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b00cf360 0x7fe2b0071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.592+0000 7fe2b744c700 1 -- 172.21.15.144:0/4057527281 >> 172.21.15.144:0/4057527281 conn(0x7fe2b00d0000 msgr2=0x7fe2b00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.592+0000 7fe2b744c700 1 -- 172.21.15.144:0/4057527281 shutdown_connections 2023-11-28T11:53:49.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.592+0000 7fe2b744c700 1 -- 172.21.15.144:0/4057527281 wait complete. 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 Processor -- start 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 -- start start 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 0x7fe2b01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2b00cf360 0x7fe2b0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b0154e70 0x7fe2b0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00c4b40 con 0x7fe2b0071580 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00c59e0 con 0x7fe2b00cf360 2023-11-28T11:53:49.598 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.593+0000 7fe2b744c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00c50b0 con 0x7fe2b0154e70 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 0x7fe2b01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 0x7fe2b01545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38264/0 (socket says 172.21.15.144:38264) 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 -- 172.21.15.144:0/2344647362 learned_addr learned my addr 172.21.15.144:0/2344647362 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 -- 172.21.15.144:0/2344647362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b0154e70 msgr2=0x7fe2b0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b0154e70 0x7fe2b0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 -- 172.21.15.144:0/2344647362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2b00cf360 msgr2=0x7fe2b0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2b00cf360 0x7fe2b0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 -- 172.21.15.144:0/2344647362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a00094a0 con 0x7fe2b0071580 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b51e8700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 0x7fe2b01545c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000c00 tx=0x7fe2a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00038a0 con 0x7fe2b0071580 2023-11-28T11:53:49.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a001e070 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b015b9a0 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0003c60 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.594+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b015b6c0 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.595+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b00af520 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.596+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe2a000f050 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.596+0000 7fe2a5ffb700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe2940682f0 0x7fe29406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.596+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe2a0086ad0 con 0x7fe2b0071580 2023-11-28T11:53:49.600 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.596+0000 7fe2b49e7700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe2940682f0 0x7fe29406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.601 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.597+0000 7fe2b49e7700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe2940682f0 0x7fe29406a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c4ea0 tx=0x7fe2ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.600+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe2a0053680 con 0x7fe2b0071580 2023-11-28T11:53:49.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.783+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2b00cd390 con 0x7fe2b0071580 2023-11-28T11:53:49.785 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.784+0000 7fe2a5ffb700 1 -- 172.21.15.144:0/2344647362 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+14257 (secure 0 0 0) 0x7fe2a0015090 con 0x7fe2b0071580 2023-11-28T11:53:49.786 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:49.786 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":56,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:49.038512+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe2940682f0 msgr2=0x7fe29406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe2940682f0 0x7fe29406a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00c4ea0 tx=0x7fe2ac008040 comp rx=0 tx=0).stop 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 msgr2=0x7fe2b01545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 0x7fe2b01545c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000c00 tx=0x7fe2a0004750 comp rx=0 tx=0).stop 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 shutdown_connections 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b0071580 0x7fe2b01545c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.788 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe2940682f0 0x7fe29406a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe2b0154e70 0x7fe2b0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 --2- 172.21.15.144:0/2344647362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2b00cf360 0x7fe2b0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 >> 172.21.15.144:0/2344647362 conn(0x7fe2b00d0000 msgr2=0x7fe2b00adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:49.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 shutdown_connections 2023-11-28T11:53:49.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.786+0000 7fe2b744c700 1 -- 172.21.15.144:0/2344647362 wait complete. 2023-11-28T11:53:49.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:53:49.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:53:49.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.922+0000 7f39ac341700 1 Processor -- start 2023-11-28T11:53:49.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.922+0000 7f39ac341700 1 -- start start 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.923+0000 7f39ac341700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 0x7f39a40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.923+0000 7f39ac341700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a413ff00 con 0x7f39a40cd8c0 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.923+0000 7f39ac341700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a41404c0 con 0x7f39a40b2e70 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.923+0000 7f39ac341700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a4140dc0 con 0x7f39a4140950 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39aa0dd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f39a40cd8c0 0x7f39a40cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:54984/0 (socket says 172.21.15.144:54984) 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39aa0dd700 1 -- 172.21.15.144:0/3829437729 learned_addr learned my addr 172.21.15.144:0/3829437729 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a98dc700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 0x7f39a40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955997951 0 0) 0x7f39a413ff00 con 0x7f39a40cd8c0 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39900035b0 con 0x7f39a40cd8c0 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126776206 0 0) 0x7f39a41404c0 con 0x7f39a40b2e70 2023-11-28T11:53:49.928 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a413ff00 con 0x7f39a40b2e70 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2344576600 0 0) 0x7f39900035b0 con 0x7f39a40cd8c0 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 msgr2=0x7f39a40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 0x7f39a40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 >> v1:172.21.15.177:6789/0 conn(0x7f39a40b2e70 legacy=0x7f39a406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a415aa80 con 0x7f39a40cd8c0 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f3994001770 con 0x7f39a40cd8c0 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 >> v1:172.21.15.144:6789/0 conn(0x7f39a40cd8c0 legacy=0x7f39a40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 0x7f3990003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004170 0x7f3990004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3990004d90 0x7f3990005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a4140dc0 con 0x7f3990004d90 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399000bba0 con 0x7f3990004170 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399000bd60 con 0x7f39a4140950 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39aa8de700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3990004d90 0x7f3990005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39a98dc700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004170 0x7f3990004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.924+0000 7f39aa0dd700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 0x7f3990003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 -- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 msgr2=0x7f3990003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a4140950 0x7f3990003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 -- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004170 msgr2=0x7f3990004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004170 0x7f3990004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 -- 172.21.15.144:0/3829437729 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39900035b0 con 0x7f3990004d90 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3990004d90 0x7f3990005200 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f39a40ce800 tx=0x7f3990005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e007d7b8c945a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c003a00 con 0x7f3990004d90 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c003b60 con 0x7f3990004d90 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39a90db700 1 -- 172.21.15.144:0/3829437729 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c0050d0 con 0x7f3990004d90 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39ac341700 1 -- 172.21.15.144:0/3829437729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3990004d90 msgr2=0x7f3990005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39ac341700 1 --2- 172.21.15.144:0/3829437729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3990004d90 0x7f3990005200 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f39a40ce800 tx=0x7f3990005880 comp rx=0 tx=0).stop 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39aa8de700 1 -- 172.21.15.144:0/3829437729 reap_dead start 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39ac341700 1 -- 172.21.15.144:0/3829437729 shutdown_connections 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.925+0000 7f39ac341700 1 -- 172.21.15.144:0/3829437729 >> 172.21.15.144:0/3829437729 conn(0x7f39a406aed0 msgr2=0x7f39a406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.926+0000 7f39ac341700 1 -- 172.21.15.144:0/3829437729 shutdown_connections 2023-11-28T11:53:49.931 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.926+0000 7f39ac341700 1 -- 172.21.15.144:0/3829437729 wait complete. 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 Processor -- start 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 -- start start 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004d90 0x7f39a4142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a41efb40 0x7f39a4143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 0x7f39a4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a4140ff0 con 0x7f39a41effb0 2023-11-28T11:53:49.934 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a41412d0 con 0x7f3990004d90 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.927+0000 7f39ac341700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a41415b0 con 0x7f39a41efb40 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 0x7f39a4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 0x7f39a4143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38288/0 (socket says 172.21.15.144:38288) 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 -- 172.21.15.144:0/1054002405 learned_addr learned my addr 172.21.15.144:0/1054002405 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa0dd700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004d90 0x7f39a4142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 -- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a41efb40 msgr2=0x7f39a4143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39a98dc700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a41efb40 0x7f39a4143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a41efb40 0x7f39a4143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 -- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004d90 msgr2=0x7f39a4142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004d90 0x7f39a4142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 -- 172.21.15.144:0/1054002405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a40b35f0 con 0x7f39a41effb0 2023-11-28T11:53:49.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39aa8de700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 0x7f39a4143ab0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f399c004000 tx=0x7f399c001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c003a00 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c001e90 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a40b3310 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c005540 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.928+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a4140150 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.929+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a40ce5e0 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.930+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f399c02e070 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.930+0000 7f399affd700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39880682f0 0x7f398806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.930+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f399c086920 con 0x7f39a41effb0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.930+0000 7f39aa0dd700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39880682f0 0x7f398806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:49.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.931+0000 7f39aa0dd700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39880682f0 0x7f398806a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3994000c00 tx=0x7f3994009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:49.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:49.934+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f399c0534d0 con 0x7f39a41effb0 2023-11-28T11:53:50.117 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.116+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39a40d08c0 con 0x7f39a41effb0 2023-11-28T11:53:50.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.116+0000 7f399affd700 1 -- 172.21.15.144:0/1054002405 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f399c022020 con 0x7f39a41effb0 2023-11-28T11:53:50.118 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39880682f0 msgr2=0x7f398806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39880682f0 0x7f398806a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3994000c00 tx=0x7f3994009040 comp rx=0 tx=0).stop 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 msgr2=0x7f39a4143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 0x7f39a4143ab0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f399c004000 tx=0x7f399c001690 comp rx=0 tx=0).stop 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 shutdown_connections 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a41effb0 0x7f39a4143ab0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f39880682f0 0x7f398806a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3990004d90 0x7f39a4142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 --2- 172.21.15.144:0/1054002405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f39a41efb40 0x7f39a4143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 >> 172.21.15.144:0/1054002405 conn(0x7f39a406aed0 msgr2=0x7f39a40cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:50.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 shutdown_connections 2023-11-28T11:53:50.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.118+0000 7f39ac341700 1 -- 172.21.15.144:0/1054002405 wait complete. 2023-11-28T11:53:50.131 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:53:50.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.252+0000 7f1727492700 1 Processor -- start 2023-11-28T11:53:50.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1727492700 1 -- start start 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1727492700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1720161340 0x7f17200d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1727492700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f17200d2320 0x7f17200cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1727492700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200d1950 con 0x7f172015c9b0 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1727492700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200b5800 con 0x7f17200d2320 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1727492700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200b5300 con 0x7f1720161340 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f1725a2f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f17200d2320 0x7f17200cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.253+0000 7f172522e700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f172015c9b0 0x7f17200b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55004/0 (socket says 172.21.15.144:55004) 2023-11-28T11:53:50.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f172522e700 1 -- 172.21.15.144:0/1018751836 learned_addr learned my addr 172.21.15.144:0/1018751836 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1724a2d700 1 --2- 172.21.15.144:0/1018751836 >> v2:172.21.15.177:3300/0 conn(0x7f1720161340 0x7f17200d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1725a2f700 1 -- 172.21.15.144:0/1018751836 >> v2:172.21.15.177:3300/0 conn(0x7f1720161340 msgr2=0x7f17200d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1725a2f700 1 --2- 172.21.15.144:0/1018751836 >> v2:172.21.15.177:3300/0 conn(0x7f1720161340 0x7f17200d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1725a2f700 1 -- 172.21.15.144:0/1018751836 >> v1:172.21.15.144:6789/0 conn(0x7f172015c9b0 legacy=0x7f17200b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1725a2f700 1 -- 172.21.15.144:0/1018751836 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200b5520 con 0x7f17200d2320 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1717fff700 1 -- 172.21.15.144:0/1018751836 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529325474 0 0) 0x7f17200d1950 con 0x7f172015c9b0 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1725a2f700 1 --2- 172.21.15.144:0/1018751836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17200d2320 0x7f17200cd7c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f17200685c0 tx=0x7f171800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be504da91af6ffe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1717fff700 1 -- 172.21.15.144:0/1018751836 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171800e070 con 0x7f17200d2320 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1717fff700 1 -- 172.21.15.144:0/1018751836 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171800bae0 con 0x7f17200d2320 2023-11-28T11:53:50.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.254+0000 7f1717fff700 1 -- 172.21.15.144:0/1018751836 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171800be90 con 0x7f17200d2320 2023-11-28T11:53:50.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 -- 172.21.15.144:0/1018751836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17200d2320 msgr2=0x7f17200cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 --2- 172.21.15.144:0/1018751836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17200d2320 0x7f17200cd7c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f17200685c0 tx=0x7f171800b330 comp rx=0 tx=0).stop 2023-11-28T11:53:50.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 -- 172.21.15.144:0/1018751836 shutdown_connections 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 --2- 172.21.15.144:0/1018751836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17200d2320 0x7f17200cd7c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 --2- 172.21.15.144:0/1018751836 >> v2:172.21.15.177:3300/0 conn(0x7f1720161340 0x7f17200d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 -- 172.21.15.144:0/1018751836 >> 172.21.15.144:0/1018751836 conn(0x7f1720072600 msgr2=0x7f17200cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 -- 172.21.15.144:0/1018751836 shutdown_connections 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.255+0000 7f1727492700 1 -- 172.21.15.144:0/1018751836 wait complete. 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.256+0000 7f1727492700 1 Processor -- start 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.256+0000 7f1727492700 1 -- start start 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17200d2320 0x7f172015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 0x7f17201558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720161340 0x7f1720155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201419b0 con 0x7f172015c9b0 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1720140da0 con 0x7f17200d2320 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1720141010 con 0x7f1720161340 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 0x7f17201558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 0x7f17201558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38302/0 (socket says 172.21.15.144:38302) 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 -- 172.21.15.144:0/3268376341 learned_addr learned my addr 172.21.15.144:0/3268376341 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f172522e700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17200d2320 0x7f172015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1725a2f700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720161340 0x7f1720155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 -- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720161340 msgr2=0x7f1720155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720161340 0x7f1720155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 -- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17200d2320 msgr2=0x7f172015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17200d2320 0x7f172015e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 -- 172.21.15.144:0/3268376341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009a70 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f172522e700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17200d2320 0x7f172015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1724a2d700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 0x7f17201558c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f171c00e9e0 tx=0x7f171c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c009710 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c00c850 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17200d15e0 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.257+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c00cbc0 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.258+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17200d1300 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.259+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172015cd80 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.259+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f171c019030 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.259+0000 7f1715ffb700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f17040682f0 0x7f170406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.259+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f171c085960 con 0x7f172015c9b0 2023-11-28T11:53:50.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.259+0000 7f172522e700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f17040682f0 0x7f170406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.260+0000 7f172522e700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f17040682f0 0x7f170406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1710000f60 tx=0x7f1710007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.264+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f171c052510 con 0x7f172015c9b0 2023-11-28T11:53:50.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.444+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1720141af0 con 0x7f172015c9b0 2023-11-28T11:53:50.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.445+0000 7f1715ffb700 1 -- 172.21.15.144:0/3268376341 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1720141af0 con 0x7f172015c9b0 2023-11-28T11:53:50.447 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:53:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f17040682f0 msgr2=0x7f170406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f17040682f0 0x7f170406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1710000f60 tx=0x7f1710007d30 comp rx=0 tx=0).stop 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 msgr2=0x7f17201558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 0x7f17201558c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f171c00e9e0 tx=0x7f171c00eda0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 shutdown_connections 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f17040682f0 0x7f170406a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720161340 0x7f1720155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17200d2320 0x7f172015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 --2- 172.21.15.144:0/3268376341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f172015c9b0 0x7f17201558c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 >> 172.21.15.144:0/3268376341 conn(0x7f1720072600 msgr2=0x7f172015fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 shutdown_connections 2023-11-28T11:53:50.450 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.447+0000 7f1727492700 1 -- 172.21.15.144:0/3268376341 wait complete. 2023-11-28T11:53:50.463 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:53:50.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.581+0000 7fa640b05700 1 Processor -- start 2023-11-28T11:53:50.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.581+0000 7fa640b05700 1 -- start start 2023-11-28T11:53:50.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa640b05700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa63c06dee0 0x7fa63c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa640b05700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa63c160540 0x7fa63c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa640b05700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c15d060 con 0x7fa63c071040 2023-11-28T11:53:50.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa640b05700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c15d340 con 0x7fa63c160540 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa640b05700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c15d620 con 0x7fa63c06dee0 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa63ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa63c160540 0x7fa63c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa63ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa63c160540 0x7fa63c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38310/0 (socket says 172.21.15.144:38310) 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa63ad9d700 1 -- 172.21.15.144:0/2822181536 learned_addr learned my addr 172.21.15.144:0/2822181536 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa639d9b700 1 --2- 172.21.15.144:0/2822181536 >> v2:172.21.15.177:3300/0 conn(0x7fa63c06dee0 0x7fa63c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa63ad9d700 1 -- 172.21.15.144:0/2822181536 >> v2:172.21.15.177:3300/0 conn(0x7fa63c06dee0 msgr2=0x7fa63c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.582+0000 7fa63ad9d700 1 --2- 172.21.15.144:0/2822181536 >> v2:172.21.15.177:3300/0 conn(0x7fa63c06dee0 0x7fa63c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa63ad9d700 1 -- 172.21.15.144:0/2822181536 >> v1:172.21.15.177:6789/0 conn(0x7fa63c071040 legacy=0x7fa63c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa63ad9d700 1 -- 172.21.15.144:0/2822181536 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c15d900 con 0x7fa63c160540 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa63ad9d700 1 --2- 172.21.15.144:0/2822181536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c160540 0x7fa63c160980 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa63c06fb00 tx=0x7fa63400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c453b5cb9642ef2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa63959a700 1 -- 172.21.15.144:0/2822181536 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400e070 con 0x7fa63c160540 2023-11-28T11:53:50.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa63959a700 1 -- 172.21.15.144:0/2822181536 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63400b9c0 con 0x7fa63c160540 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa63959a700 1 -- 172.21.15.144:0/2822181536 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63400bd70 con 0x7fa63c160540 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa640b05700 1 -- 172.21.15.144:0/2822181536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c160540 msgr2=0x7fa63c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa640b05700 1 --2- 172.21.15.144:0/2822181536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c160540 0x7fa63c160980 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa63c06fb00 tx=0x7fa63400b330 comp rx=0 tx=0).stop 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa640b05700 1 -- 172.21.15.144:0/2822181536 shutdown_connections 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa640b05700 1 --2- 172.21.15.144:0/2822181536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c160540 0x7fa63c160980 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa640b05700 1 --2- 172.21.15.144:0/2822181536 >> v2:172.21.15.177:3300/0 conn(0x7fa63c06dee0 0x7fa63c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.583+0000 7fa640b05700 1 -- 172.21.15.144:0/2822181536 >> 172.21.15.144:0/2822181536 conn(0x7fa63c0d0000 msgr2=0x7fa63c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.584+0000 7fa640b05700 1 -- 172.21.15.144:0/2822181536 shutdown_connections 2023-11-28T11:53:50.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.584+0000 7fa640b05700 1 -- 172.21.15.144:0/2822181536 wait complete. 2023-11-28T11:53:50.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.585+0000 7fa640b05700 1 Processor -- start 2023-11-28T11:53:50.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.585+0000 7fa640b05700 1 -- start start 2023-11-28T11:53:50.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 0x7fa63c150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa63c071040 0x7fa63c0c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa63c160540 0x7fa63c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0c3e20 con 0x7fa63c06dee0 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0c4100 con 0x7fa63c071040 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0c43e0 con 0x7fa63c160540 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 0x7fa63c150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 0x7fa63c150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38312/0 (socket says 172.21.15.144:38312) 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 -- 172.21.15.144:0/1005181127 learned_addr learned my addr 172.21.15.144:0/1005181127 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 -- 172.21.15.144:0/1005181127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa63c160540 msgr2=0x7fa63c0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa63c160540 0x7fa63c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 -- 172.21.15.144:0/1005181127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa63c071040 msgr2=0x7fa63c0c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa63c071040 0x7fa63c0c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 -- 172.21.15.144:0/1005181127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634009b30 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa63a59c700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 0x7fa63c150bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa63000c450 tx=0x7fa63000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300052d0 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630005430 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c15ccd0 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300038c0 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.586+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c06e870 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.587+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c06a270 con 0x7fa63c06dee0 2023-11-28T11:53:50.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.588+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa630019030 con 0x7fa63c06dee0 2023-11-28T11:53:50.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.588+0000 7fa62affd700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa61c0682f0 0x7fa61c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.588+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa630084cd0 con 0x7fa63c06dee0 2023-11-28T11:53:50.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.588+0000 7fa639d9b700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa61c0682f0 0x7fa61c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.589+0000 7fa639d9b700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa61c0682f0 0x7fa61c06a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0cc390 tx=0x7fa62c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.592+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa630051920 con 0x7fa63c06dee0 2023-11-28T11:53:50.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.776+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa63c0720a0 con 0x7fa63c06dee0 2023-11-28T11:53:50.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.777+0000 7fa62affd700 1 -- 172.21.15.144:0/1005181127 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa630054f40 con 0x7fa63c06dee0 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout:[ 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 1, 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": "rbd" 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: "poolnum": 2, 2023-11-28T11:53:50.779 INFO:teuthology.orchestra.run.smithi144.stdout: "poolname": ".mgr" 2023-11-28T11:53:50.780 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:53:50.780 INFO:teuthology.orchestra.run.smithi144.stdout:] 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa61c0682f0 msgr2=0x7fa61c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa61c0682f0 0x7fa61c06a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0cc390 tx=0x7fa62c008040 comp rx=0 tx=0).stop 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 msgr2=0x7fa63c150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 0x7fa63c150bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa63000c450 tx=0x7fa63000c810 comp rx=0 tx=0).stop 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 shutdown_connections 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa63c06dee0 0x7fa63c150bc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa61c0682f0 0x7fa61c06a7a0 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0cc390 tx=0x7fa62c008040 comp rx=0 tx=0).stop 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa63c160540 0x7fa63c0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 --2- 172.21.15.144:0/1005181127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa63c071040 0x7fa63c0c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.781 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 >> 172.21.15.144:0/1005181127 conn(0x7fa63c0d0000 msgr2=0x7fa63c15fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:50.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 shutdown_connections 2023-11-28T11:53:50.782 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.779+0000 7fa640b05700 1 -- 172.21.15.144:0/1005181127 wait complete. 2023-11-28T11:53:50.793 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:53:50.793 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:53:50.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.915+0000 7f1079ca8700 1 Processor -- start 2023-11-28T11:53:50.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.915+0000 7f1079ca8700 1 -- start start 2023-11-28T11:53:50.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1079ca8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10740ce7c0 0x7f10740b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1079ca8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10740cb800 0x7f1074163b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.919 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1079ca8700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107406bb50 con 0x7f107406ff30 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1079ca8700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10740cf310 con 0x7f10740ce7c0 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1079ca8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107406b590 con 0x7f10740cb800 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f10737fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10740ce7c0 0x7f10740b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f106bfff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f107406ff30 0x7f10740b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55034/0 (socket says 172.21.15.144:55034) 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f106bfff700 1 -- 172.21.15.144:0/3612150209 learned_addr learned my addr 172.21.15.144:0/3612150209 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1073fff700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10740cb800 0x7f1074163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f10737fe700 1 -- 172.21.15.144:0/3612150209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10740cb800 msgr2=0x7f1074163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f10737fe700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10740cb800 0x7f1074163b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f10737fe700 1 -- 172.21.15.144:0/3612150209 >> v1:172.21.15.144:6789/0 conn(0x7f107406ff30 legacy=0x7f10740b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1072ffd700 1 -- 172.21.15.144:0/3612150209 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251564280 0 0) 0x7f107406bb50 con 0x7f107406ff30 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f10737fe700 1 -- 172.21.15.144:0/3612150209 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107406b870 con 0x7f10740ce7c0 2023-11-28T11:53:50.920 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.916+0000 7f1073fff700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10740cb800 0x7f1074163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f10737fe700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740ce7c0 0x7f10740b4470 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1064009840 tx=0x7f1064009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7127d4dd97729cd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1072ffd700 1 -- 172.21.15.144:0/3612150209 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064011830 con 0x7f10740ce7c0 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1072ffd700 1 -- 172.21.15.144:0/3612150209 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064011990 con 0x7f10740ce7c0 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1072ffd700 1 -- 172.21.15.144:0/3612150209 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064011d20 con 0x7f10740ce7c0 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 -- 172.21.15.144:0/3612150209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740ce7c0 msgr2=0x7f10740b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740ce7c0 0x7f10740b4470 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1064009840 tx=0x7f1064009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 -- 172.21.15.144:0/3612150209 shutdown_connections 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f10740cb800 0x7f1074163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3612150209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740ce7c0 0x7f10740b4470 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 -- 172.21.15.144:0/3612150209 >> 172.21.15.144:0/3612150209 conn(0x7f10740b3080 msgr2=0x7f10740cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 -- 172.21.15.144:0/3612150209 shutdown_connections 2023-11-28T11:53:50.921 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.917+0000 7f1079ca8700 1 -- 172.21.15.144:0/3612150209 wait complete. 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.919+0000 7f1079ca8700 1 Processor -- start 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.919+0000 7f1079ca8700 1 -- start start 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1079ca8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107406ff30 0x7f10741ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1079ca8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 0x7f10740c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1079ca8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10740ce7c0 0x7f107406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1079ca8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10740adbf0 con 0x7f10740cb800 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1079ca8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107406d7e0 con 0x7f10740ce7c0 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1079ca8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10741ee6f0 con 0x7f107406ff30 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 0x7f10740c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 0x7f10740c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38328/0 (socket says 172.21.15.144:38328) 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 -- 172.21.15.144:0/3051655332 learned_addr learned my addr 172.21.15.144:0/3051655332 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:50.924 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f10737fe700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107406ff30 0x7f10741ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f1073fff700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10740ce7c0 0x7f107406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 -- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107406ff30 msgr2=0x7f10741ddb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107406ff30 0x7f10741ddb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 -- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10740ce7c0 msgr2=0x7f107406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10740ce7c0 0x7f107406d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 -- 172.21.15.144:0/3051655332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640094a0 con 0x7f10740cb800 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.920+0000 7f106bfff700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 0x7f10740c8f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c550 tx=0x7f105c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.921+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c004c80 con 0x7f10740cb800 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.921+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c004de0 con 0x7f10740cb800 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.921+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10740d0200 con 0x7f10740cb800 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.921+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0072d0 con 0x7f10740cb800 2023-11-28T11:53:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.921+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10740cff20 con 0x7f10740cb800 2023-11-28T11:53:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.922+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107406a650 con 0x7f10740cb800 2023-11-28T11:53:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.922+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f105c01a020 con 0x7f10740cb800 2023-11-28T11:53:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.923+0000 7f1070ff9700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10540682f0 0x7f105406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.923+0000 7f10737fe700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10540682f0 0x7f105406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.923+0000 7f10737fe700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10540682f0 0x7f105406a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10640090b0 tx=0x7f1064008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:50.927 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.923+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f105c05dc60 con 0x7f10740cb800 2023-11-28T11:53:50.929 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:50.928+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f105c051f10 con 0x7f10740cb800 2023-11-28T11:53:51.114 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.112+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f10740add30 con 0x7f10740cb800 2023-11-28T11:53:51.930 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.928+0000 7f1070ff9700 1 -- 172.21.15.144:0/3051655332 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f10740add30 con 0x7f10740cb800 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10540682f0 msgr2=0x7f105406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10540682f0 0x7f105406a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f10640090b0 tx=0x7f1064008030 comp rx=0 tx=0).stop 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 msgr2=0x7f10740c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 0x7f10740c8f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c550 tx=0x7f105c00c860 comp rx=0 tx=0).stop 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 shutdown_connections 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f10540682f0 0x7f105406a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10740ce7c0 0x7f107406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f107406ff30 0x7f10741ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 --2- 172.21.15.144:0/3051655332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10740cb800 0x7f10740c8f60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 >> 172.21.15.144:0/3051655332 conn(0x7f10740b3080 msgr2=0x7f10740cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:51.935 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 shutdown_connections 2023-11-28T11:53:51.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:51.933+0000 7f1079ca8700 1 -- 172.21.15.144:0/3051655332 wait complete. 2023-11-28T11:53:51.937 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' created 2023-11-28T11:53:51.951 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:53:52.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.082+0000 7f2d226ad700 1 Processor -- start 2023-11-28T11:53:52.084 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.082+0000 7f2d226ad700 1 -- start start 2023-11-28T11:53:52.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d226ad700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2d1c071040 0x7f2d1c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d226ad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d1c06dee0 0x7f2d1c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d226ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06cdc0 0x7f2d1c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d226ad700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1c071950 con 0x7f2d1c06dee0 2023-11-28T11:53:52.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d226ad700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1c15d060 con 0x7f2d1c071040 2023-11-28T11:53:52.086 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d226ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c15d340 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d1c06dee0 0x7f2d1c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d1c06dee0 0x7f2d1c06e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38334/0 (socket says 172.21.15.144:38334) 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 -- 172.21.15.144:0/3626107943 learned_addr learned my addr 172.21.15.144:0/3626107943 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d20c4a700 1 --2- 172.21.15.144:0/3626107943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06cdc0 0x7f2d1c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1bfff700 1 --2- 172.21.15.144:0/3626107943 >> v2:172.21.15.177:3300/0 conn(0x7f2d1c071040 0x7f2d1c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 -- 172.21.15.144:0/3626107943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06cdc0 msgr2=0x7f2d1c06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 --2- 172.21.15.144:0/3626107943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06cdc0 0x7f2d1c06d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 -- 172.21.15.144:0/3626107943 >> v2:172.21.15.177:3300/0 conn(0x7f2d1c071040 msgr2=0x7f2d1c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 --2- 172.21.15.144:0/3626107943 >> v2:172.21.15.177:3300/0 conn(0x7f2d1c071040 0x7f2d1c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.083+0000 7f2d1b7fe700 1 -- 172.21.15.144:0/3626107943 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c15d620 con 0x7f2d1c06dee0 2023-11-28T11:53:52.087 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d1b7fe700 1 --2- 172.21.15.144:0/3626107943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06dee0 0x7f2d1c06e330 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0d0f50 tx=0x7f2d0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f7e2ff4ad32019 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d1affd700 1 -- 172.21.15.144:0/3626107943 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00b870 con 0x7f2d1c06dee0 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d1affd700 1 -- 172.21.15.144:0/3626107943 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c00b9d0 con 0x7f2d1c06dee0 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d1affd700 1 -- 172.21.15.144:0/3626107943 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00bd60 con 0x7f2d1c06dee0 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 -- 172.21.15.144:0/3626107943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06dee0 msgr2=0x7f2d1c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 --2- 172.21.15.144:0/3626107943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06dee0 0x7f2d1c06e330 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0d0f50 tx=0x7f2d0c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 -- 172.21.15.144:0/3626107943 shutdown_connections 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 --2- 172.21.15.144:0/3626107943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06cdc0 0x7f2d1c06d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 --2- 172.21.15.144:0/3626107943 >> v2:172.21.15.177:3300/0 conn(0x7f2d1c071040 0x7f2d1c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 --2- 172.21.15.144:0/3626107943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06dee0 0x7f2d1c06e330 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 -- 172.21.15.144:0/3626107943 >> 172.21.15.144:0/3626107943 conn(0x7f2d1c0d0000 msgr2=0x7f2d1c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:52.088 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 -- 172.21.15.144:0/3626107943 shutdown_connections 2023-11-28T11:53:52.089 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.084+0000 7f2d226ad700 1 -- 172.21.15.144:0/3626107943 wait complete. 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 Processor -- start 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 -- start start 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 0x7f2d1c0c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06dee0 0x7f2d1c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1c071040 0x7f2d1c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c158ac0 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0c4230 con 0x7f2d1c071040 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.086+0000 7f2d226ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0c4510 con 0x7f2d1c06dee0 2023-11-28T11:53:52.091 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 0x7f2d1c0c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 0x7f2d1c0c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38340/0 (socket says 172.21.15.144:38340) 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 -- 172.21.15.144:0/2685222896 learned_addr learned my addr 172.21.15.144:0/2685222896 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 -- 172.21.15.144:0/2685222896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06dee0 msgr2=0x7f2d1c158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06dee0 0x7f2d1c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 -- 172.21.15.144:0/2685222896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1c071040 msgr2=0x7f2d1c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1c071040 0x7f2d1c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 -- 172.21.15.144:0/2685222896 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c009550 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d1bfff700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 0x7f2d1c0c3b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000d800 tx=0x7f2d1000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d100125f0 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10012750 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c06e870 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.092 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10012b00 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.087+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c15d900 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.088+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1c15ccd0 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.088+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d10012c60 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.089+0000 7f2d18ff9700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d080682f0 0x7f2d0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:52.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.089+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f2d10085fa0 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.093 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.089+0000 7f2d1b7fe700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d080682f0 0x7f2d0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:52.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.089+0000 7f2d1b7fe700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d080682f0 0x7f2d0806a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0b5ad0 tx=0x7f2d0c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:52.095 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.093+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d100529e0 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.275+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2d1c0720a0 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.936 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.932+0000 7f2d18ff9700 1 -- 172.21.15.144:0/2685222896 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2d10056000 con 0x7f2d1c06cdc0 2023-11-28T11:53:52.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.941+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d080682f0 msgr2=0x7f2d0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:52.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.941+0000 7f2d226ad700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d080682f0 0x7f2d0806a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0b5ad0 tx=0x7f2d0c010000 comp rx=0 tx=0).stop 2023-11-28T11:53:52.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.941+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 msgr2=0x7f2d1c0c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:52.943 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.941+0000 7f2d226ad700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 0x7f2d1c0c3b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000d800 tx=0x7f2d1000dbc0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.942+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 shutdown_connections 2023-11-28T11:53:52.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.942+0000 7f2d226ad700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c06cdc0 0x7f2d1c0c3b50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.942+0000 7f2d226ad700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d080682f0 0x7f2d0806a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.942+0000 7f2d226ad700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d1c071040 0x7f2d1c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.942+0000 7f2d226ad700 1 --2- 172.21.15.144:0/2685222896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d1c06dee0 0x7f2d1c158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:52.944 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.942+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 >> 172.21.15.144:0/2685222896 conn(0x7f2d1c0d0000 msgr2=0x7f2d1c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:52.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.943+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 shutdown_connections 2023-11-28T11:53:52.945 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:52.943+0000 7f2d226ad700 1 -- 172.21.15.144:0/2685222896 wait complete. 2023-11-28T11:53:52.947 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' created 2023-11-28T11:53:52.966 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:53:52.984 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:53:53.011 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:53:53.097 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.096+0000 7fc692ac2700 1 Processor -- start 2023-11-28T11:53:53.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.096+0000 7fc692ac2700 1 -- start start 2023-11-28T11:53:53.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc692ac2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc68c15cef0 0x7fc68c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc692ac2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 0x7fc68c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc692ac2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68c0cc5f0 con 0x7fc68c06b8c0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc692ac2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc68c06bd40 con 0x7fc68c15cef0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc692ac2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c15d470 con 0x7fc68c0cbdf0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc69105f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc68c06b8c0 0x7fc68c071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55064/0 (socket says 172.21.15.144:55064) 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc69105f700 1 -- 172.21.15.144:0/3182939792 learned_addr learned my addr 172.21.15.144:0/3182939792 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc69085e700 1 --2- 172.21.15.144:0/3182939792 >> v2:172.21.15.177:3300/0 conn(0x7fc68c15cef0 0x7fc68c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68bfff700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 0x7fc68c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784056428 0 0) 0x7fc68c0cc5f0 con 0x7fc68c06b8c0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c0035b0 con 0x7fc68c06b8c0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4084379127 0 0) 0x7fc67c0035b0 con 0x7fc68c06b8c0 2023-11-28T11:53:53.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 msgr2=0x7fc68c06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 0x7fc68c06b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 >> v2:172.21.15.177:3300/0 conn(0x7fc68c15cef0 msgr2=0x7fc68c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 --2- 172.21.15.144:0/3182939792 >> v2:172.21.15.177:3300/0 conn(0x7fc68c15cef0 0x7fc68c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c15d750 con 0x7fc68c06b8c0 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fc6840019f0 con 0x7fc68c06b8c0 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc69085e700 1 --2- 172.21.15.144:0/3182939792 >> v2:172.21.15.177:3300/0 conn(0x7fc68c15cef0 0x7fc68c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 >> v1:172.21.15.144:6789/0 conn(0x7fc68c06b8c0 legacy=0x7fc68c071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 0x7fc67c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c003f80 0x7fc67c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004d90 0x7fc67c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c06bd40 con 0x7fc67c004d90 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c15d470 con 0x7fc67c003f80 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c00ba10 con 0x7fc68c0cbdf0 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.097+0000 7fc69105f700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004d90 0x7fc67c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc68bfff700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c003f80 0x7fc67c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69085e700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 0x7fc67c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69105f700 1 -- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 msgr2=0x7fc67c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69105f700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c0cbdf0 0x7fc67c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69105f700 1 -- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c003f80 msgr2=0x7fc67c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69105f700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c003f80 0x7fc67c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69105f700 1 -- 172.21.15.144:0/3182939792 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c0035b0 con 0x7fc67c004d90 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc68bfff700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc67c003f80 0x7fc67c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:53.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc69105f700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004d90 0x7fc67c005200 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc68400a860 tx=0x7fc67c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398c14dcd12bdd36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684012600 con 0x7fc67c004d90 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684012760 con 0x7fc67c004d90 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.098+0000 7fc68b7fe700 1 -- 172.21.15.144:0/3182939792 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684012b10 con 0x7fc67c004d90 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc692ac2700 1 -- 172.21.15.144:0/3182939792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004d90 msgr2=0x7fc67c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc692ac2700 1 --2- 172.21.15.144:0/3182939792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc67c004d90 0x7fc67c005200 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc68400a860 tx=0x7fc67c005880 comp rx=0 tx=0).stop 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc69105f700 1 -- 172.21.15.144:0/3182939792 reap_dead start 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc692ac2700 1 -- 172.21.15.144:0/3182939792 shutdown_connections 2023-11-28T11:53:53.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc692ac2700 1 -- 172.21.15.144:0/3182939792 >> 172.21.15.144:0/3182939792 conn(0x7fc68c0d0000 msgr2=0x7fc68c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:53.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc692ac2700 1 -- 172.21.15.144:0/3182939792 shutdown_connections 2023-11-28T11:53:53.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.099+0000 7fc692ac2700 1 -- 172.21.15.144:0/3182939792 wait complete. 2023-11-28T11:53:53.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.100+0000 7fc692ac2700 1 Processor -- start 2023-11-28T11:53:53.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.100+0000 7fc692ac2700 1 -- start start 2023-11-28T11:53:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc692ac2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc68c156630 0x7fc68c0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc692ac2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 0x7fc68c1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc692ac2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c1f6570 0x7fc68c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc692ac2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c1f58e0 con 0x7fc68c1f22f0 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc692ac2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c1f2ad0 con 0x7fc68c156630 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc692ac2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c1f2db0 con 0x7fc68c1f6570 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 0x7fc68c1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 0x7fc68c1f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38362/0 (socket says 172.21.15.144:38362) 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 -- 172.21.15.144:0/1807313564 learned_addr learned my addr 172.21.15.144:0/1807313564 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc69105f700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c1f6570 0x7fc68c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc69085e700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc68c156630 0x7fc68c0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 -- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c1f6570 msgr2=0x7fc68c158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c1f6570 0x7fc68c158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 -- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc68c156630 msgr2=0x7fc68c0adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc68c156630 0x7fc68c0adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 -- 172.21.15.144:0/1807313564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c15dd10 con 0x7fc68c1f22f0 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc68bfff700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 0x7fc68c1f6030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc678011cd0 tx=0x7fc678009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678004020 con 0x7fc68c1f22f0 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.101+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678004180 con 0x7fc68c1f22f0 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.102+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c15da30 con 0x7fc68c1f22f0 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.102+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678004510 con 0x7fc68c1f22f0 2023-11-28T11:53:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.102+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c15dff0 con 0x7fc68c1f22f0 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.103+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68c0d0e60 con 0x7fc68c1f22f0 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.103+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc678024070 con 0x7fc68c1f22f0 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.103+0000 7fc6897fa700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6700682f0 0x7fc67006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.103+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc678086010 con 0x7fc68c1f22f0 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.103+0000 7fc69085e700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6700682f0 0x7fc67006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.104+0000 7fc69085e700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6700682f0 0x7fc67006a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc680009e60 tx=0x7fc680009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.108+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6780528d0 con 0x7fc68c1f22f0 2023-11-28T11:53:53.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.335+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc68c0af1a0 con 0x7fc68c1f22f0 2023-11-28T11:53:53.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.343+0000 7fc6897fa700 1 -- 172.21.15.144:0/1807313564 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc678022020 con 0x7fc68c1f22f0 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6700682f0 msgr2=0x7fc67006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6700682f0 0x7fc67006a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc680009e60 tx=0x7fc680009210 comp rx=0 tx=0).stop 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 msgr2=0x7fc68c1f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 0x7fc68c1f6030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc678011cd0 tx=0x7fc678009e70 comp rx=0 tx=0).stop 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 shutdown_connections 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc6700682f0 0x7fc67006a7a0 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc680009e60 tx=0x7fc680009210 comp rx=0 tx=0).stop 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc68c1f6570 0x7fc68c158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc68c156630 0x7fc68c0adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 --2- 172.21.15.144:0/1807313564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc68c1f22f0 0x7fc68c1f6030 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 >> 172.21.15.144:0/1807313564 conn(0x7fc68c0d0000 msgr2=0x7fc68c073190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:53.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.348+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 shutdown_connections 2023-11-28T11:53:53.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.349+0000 7fc692ac2700 1 -- 172.21.15.144:0/1807313564 wait complete. 2023-11-28T11:53:53.356 INFO:teuthology.orchestra.run.smithi144.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T11:53:53.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.530+0000 7f288672f700 1 Processor -- start 2023-11-28T11:53:53.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.530+0000 7f288672f700 1 -- start start 2023-11-28T11:53:53.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f288672f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2880071040 0x7f2880071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f288672f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f288006dee0 0x7f288006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f288672f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2880071950 con 0x7f288006cdc0 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f288672f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2880161550 con 0x7f288006dee0 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f288672f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288015d060 con 0x7f2880071040 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f287f7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f288006dee0 0x7f288006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f287ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2880071040 0x7f2880071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f2884ccc700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f288006cdc0 0x7f288006d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55586/0 (socket says 172.21.15.144:55586) 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.531+0000 7f2884ccc700 1 -- 172.21.15.144:0/2398070686 learned_addr learned my addr 172.21.15.144:0/2398070686 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287effd700 1 -- 172.21.15.144:0/2398070686 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606426653 0 0) 0x7f2880071950 con 0x7f288006cdc0 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287effd700 1 -- 172.21.15.144:0/2398070686 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28680035b0 con 0x7f288006cdc0 2023-11-28T11:53:53.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287f7fe700 1 -- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2880071040 msgr2=0x7f2880071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287f7fe700 1 --2- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2880071040 0x7f2880071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287f7fe700 1 -- 172.21.15.144:0/2398070686 >> v1:172.21.15.177:6789/0 conn(0x7f288006cdc0 legacy=0x7f288006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287f7fe700 1 -- 172.21.15.144:0/2398070686 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288015d340 con 0x7f288006dee0 2023-11-28T11:53:53.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287ffff700 1 --2- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2880071040 0x7f2880071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:53.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287f7fe700 1 --2- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f288006dee0 0x7f288006e350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2870009840 tx=0x7f2870009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9edf28f020ec20 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287effd700 1 -- 172.21.15.144:0/2398070686 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870011830 con 0x7f288006dee0 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287effd700 1 -- 172.21.15.144:0/2398070686 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2870011990 con 0x7f288006dee0 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f287effd700 1 -- 172.21.15.144:0/2398070686 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870011d20 con 0x7f288006dee0 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f288672f700 1 -- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f288006dee0 msgr2=0x7f288006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.532+0000 7f288672f700 1 --2- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f288006dee0 0x7f288006e350 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2870009840 tx=0x7f2870009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.533+0000 7f288672f700 1 -- 172.21.15.144:0/2398070686 shutdown_connections 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.533+0000 7f288672f700 1 --2- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2880071040 0x7f2880071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.533+0000 7f288672f700 1 --2- 172.21.15.144:0/2398070686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f288006dee0 0x7f288006e350 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.533+0000 7f288672f700 1 -- 172.21.15.144:0/2398070686 >> 172.21.15.144:0/2398070686 conn(0x7f28800d0000 msgr2=0x7f28800d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.533+0000 7f288672f700 1 -- 172.21.15.144:0/2398070686 shutdown_connections 2023-11-28T11:53:53.537 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.533+0000 7f288672f700 1 -- 172.21.15.144:0/2398070686 wait complete. 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.534+0000 7f288672f700 1 Processor -- start 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.534+0000 7f288672f700 1 -- start start 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 0x7f28800b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f288006dee0 0x7f28800b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880071040 0x7f2880155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2880155ff0 con 0x7f288006cdc0 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28800c4b70 con 0x7f2880071040 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28800c6b20 con 0x7f288006dee0 2023-11-28T11:53:53.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 0x7f28800b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 0x7f28800b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38386/0 (socket says 172.21.15.144:38386) 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 -- 172.21.15.144:0/112221281 learned_addr learned my addr 172.21.15.144:0/112221281 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287f7fe700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f288006dee0 0x7f28800b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f2884ccc700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880071040 0x7f2880155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 -- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f288006dee0 msgr2=0x7f28800b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f288006dee0 0x7f28800b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 -- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880071040 msgr2=0x7f2880155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880071040 0x7f2880155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 -- 172.21.15.144:0/112221281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740094a0 con 0x7f288006cdc0 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287ffff700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 0x7f28800b3840 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f287400ec10 tx=0x7f2874009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400c8e0 con 0x7f288006cdc0 2023-11-28T11:53:53.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400ca40 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28700094a0 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400cdd0 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.535+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288015d900 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.536+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288015d620 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.537+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2874019030 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.537+0000 7f287cff9700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f286c0682f0 0x7f286c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.537+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2874085cd0 con 0x7f288006cdc0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.537+0000 7f287f7fe700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f286c0682f0 0x7f286c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.542 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.538+0000 7f287f7fe700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f286c0682f0 0x7f286c06a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2870000c00 tx=0x7f287001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.543 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.542+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2874052530 con 0x7f288006cdc0 2023-11-28T11:53:53.732 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.730+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28801565b0 con 0x7f288006cdc0 2023-11-28T11:53:53.733 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.731+0000 7f287cff9700 1 -- 172.21.15.144:0/112221281 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17924 (secure 0 0 0) 0x7f2874055b50 con 0x7f288006cdc0 2023-11-28T11:53:53.733 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:53.734 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":59,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:53.337545+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f286c0682f0 msgr2=0x7f286c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f286c0682f0 0x7f286c06a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2870000c00 tx=0x7f287001c040 comp rx=0 tx=0).stop 2023-11-28T11:53:53.736 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 msgr2=0x7f28800b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 0x7f28800b3840 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f287400ec10 tx=0x7f2874009e20 comp rx=0 tx=0).stop 2023-11-28T11:53:53.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 shutdown_connections 2023-11-28T11:53:53.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f288006cdc0 0x7f28800b3840 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f286c0682f0 0x7f286c06a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2880071040 0x7f2880155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 --2- 172.21.15.144:0/112221281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f288006dee0 0x7f28800b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 >> 172.21.15.144:0/112221281 conn(0x7f28800d0000 msgr2=0x7f28800cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:53.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 shutdown_connections 2023-11-28T11:53:53.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.733+0000 7f288672f700 1 -- 172.21.15.144:0/112221281 wait complete. 2023-11-28T11:53:53.748 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:53.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.866+0000 7fa7c0a22700 1 Processor -- start 2023-11-28T11:53:53.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.867+0000 7fa7c0a22700 1 -- start start 2023-11-28T11:53:53.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.867+0000 7fa7c0a22700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa7bc06c990 0x7fa7bc06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.867+0000 7fa7c0a22700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc06ae60 con 0x7fa7bc0ca240 2023-11-28T11:53:53.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.867+0000 7fa7c0a22700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc06a270 con 0x7fa7bc06e9f0 2023-11-28T11:53:53.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7c0a22700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc06d350 con 0x7fa7bc06c990 2023-11-28T11:53:53.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7bad9d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa7bc0ca240 0x7fa7bc0ca630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55110/0 (socket says 172.21.15.144:55110) 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7bad9d700 1 -- 172.21.15.144:0/3789736083 learned_addr learned my addr 172.21.15.144:0/3789736083 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/3789736083 >> v2:172.21.15.177:3300/0 conn(0x7fa7bc06c990 0x7fa7bc06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24359936 0 0) 0x7fa7bc06ae60 con 0x7fa7bc0ca240 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a40035b0 con 0x7fa7bc0ca240 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953027637 0 0) 0x7fa7a40035b0 con 0x7fa7bc0ca240 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 >> v2:172.21.15.177:3300/0 conn(0x7fa7bc06c990 msgr2=0x7fa7bc06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 --2- 172.21.15.144:0/3789736083 >> v2:172.21.15.177:3300/0 conn(0x7fa7bc06c990 0x7fa7bc06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 >> v1:172.21.15.177:6789/0 conn(0x7fa7bc06e9f0 legacy=0x7fa7bc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc06a530 con 0x7fa7bc0ca240 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fa7b40019f0 con 0x7fa7bc0ca240 2023-11-28T11:53:53.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 >> v1:172.21.15.144:6789/0 conn(0x7fa7bc0ca240 legacy=0x7fa7bc0ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a4005200 0x7fa7a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc06d350 con 0x7fa7a4004850 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400c190 con 0x7fa7a4005200 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400c350 con 0x7fa7a4003a00 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.868+0000 7fa7ba59c700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7bad9d700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a4005200 0x7fa7a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b9d9b700 1 -- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a4003a00 msgr2=0x7fa7a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b9d9b700 1 -- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a4005200 msgr2=0x7fa7a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a4005200 0x7fa7a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b9d9b700 1 -- 172.21.15.144:0/3789736083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40035b0 con 0x7fa7a4004850 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7bad9d700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7a4005200 0x7fa7a400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7bad9d700 1 -- 172.21.15.144:0/3789736083 reap_dead start 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7a4004cc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00be30 tx=0x7fa7a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2ada9392405d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac012040 con 0x7fa7a4004850 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac00ddc0 con 0x7fa7a4004850 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7b959a700 1 -- 172.21.15.144:0/3789736083 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0185f0 con 0x7fa7a4004850 2023-11-28T11:53:53.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7c0a22700 1 -- 172.21.15.144:0/3789736083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 msgr2=0x7fa7a4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.869+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7a4004cc0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00be30 tx=0x7fa7a4005880 comp rx=0 tx=0).stop 2023-11-28T11:53:53.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.870+0000 7fa7c0a22700 1 -- 172.21.15.144:0/3789736083 shutdown_connections 2023-11-28T11:53:53.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.870+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/3789736083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7a4004cc0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.870+0000 7fa7c0a22700 1 -- 172.21.15.144:0/3789736083 >> 172.21.15.144:0/3789736083 conn(0x7fa7bc0b12b0 msgr2=0x7fa7bc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:53.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.870+0000 7fa7c0a22700 1 -- 172.21.15.144:0/3789736083 shutdown_connections 2023-11-28T11:53:53.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.870+0000 7fa7c0a22700 1 -- 172.21.15.144:0/3789736083 wait complete. 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.871+0000 7fa7c0a22700 1 Processor -- start 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.871+0000 7fa7c0a22700 1 -- start start 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7bc1f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7bc0b25f0 0x7fa7bc0c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc0b02c0 0x7fa7bc0b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b0c70 con 0x7fa7a4004850 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b37c0 con 0x7fa7bc0b02c0 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b2bb0 con 0x7fa7bc0b25f0 2023-11-28T11:53:53.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7bc1f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7bc1f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38406/0 (socket says 172.21.15.144:38406) 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 -- 172.21.15.144:0/4213565478 learned_addr learned my addr 172.21.15.144:0/4213565478 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7bad9d700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc0b02c0 0x7fa7bc0b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7bc0b25f0 0x7fa7bc0c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 -- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7bc0b25f0 msgr2=0x7fa7bc0c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7bc0b25f0 0x7fa7bc0c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 -- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc0b02c0 msgr2=0x7fa7bc0b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc0b02c0 0x7fa7bc0b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 -- 172.21.15.144:0/4213565478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc06aaf0 con 0x7fa7a4004850 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7ba59c700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7bc1f13b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000f820 tx=0x7fa7b000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00052e0 con 0x7fa7a4004850 2023-11-28T11:53:53.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0005440 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00038c0 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc06a810 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.872+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc06edc0 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.873+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc0cb470 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.874+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7b0003a20 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.874+0000 7fa7aaffd700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa79c068320 0x7fa79c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.874+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7b0085c40 con 0x7fa7a4004850 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.874+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa79c068320 0x7fa79c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:53.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.874+0000 7fa7b9d9b700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa79c068320 0x7fa79c06a7d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:53.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:53.878+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7b0052410 con 0x7fa7a4004850 2023-11-28T11:53:53.923 INFO:tasks.ceph.mds.a.smithi144.stderr:2023-11-28T11:53:53.922+0000 7ff9ac7e3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:53:54.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.063+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7bc0b5400 con 0x7fa7a4004850 2023-11-28T11:53:54.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.064+0000 7fa7aaffd700 1 -- 172.21.15.144:0/4213565478 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+17924 (secure 0 0 0) 0x7fa7b0055a30 con 0x7fa7a4004850 2023-11-28T11:53:54.066 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:54.067 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":59,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:53.337545+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:54.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa79c068320 msgr2=0x7fa79c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:54.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa79c068320 0x7fa79c06a7d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac013000 comp rx=0 tx=0).stop 2023-11-28T11:53:54.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 msgr2=0x7fa7bc1f13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:54.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7bc1f13b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000f820 tx=0x7fa7b000fbe0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 shutdown_connections 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7a4004850 0x7fa7bc1f13b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa79c068320 0x7fa79c06a7d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7bc0b02c0 0x7fa7bc0b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 --2- 172.21.15.144:0/4213565478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7bc0b25f0 0x7fa7bc0c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 >> 172.21.15.144:0/4213565478 conn(0x7fa7bc0b12b0 msgr2=0x7fa7bc0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.066+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 shutdown_connections 2023-11-28T11:53:54.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.067+0000 7fa7c0a22700 1 -- 172.21.15.144:0/4213565478 wait complete. 2023-11-28T11:53:54.081 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:53:54.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.203+0000 7f94ef568700 1 Processor -- start 2023-11-28T11:53:54.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.203+0000 7f94ef568700 1 -- start start 2023-11-28T11:53:54.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ef568700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94e815cef0 0x7f94e80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:54.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ef568700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e80cbdf0 0x7f94e806b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ef568700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e80cc5f0 con 0x7f94e806b8c0 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ef568700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e806bd40 con 0x7f94e815cef0 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ef568700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e815d470 con 0x7f94e80cbdf0 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94e815cef0 0x7f94e80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94e815cef0 0x7f94e80af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38414/0 (socket says 172.21.15.144:38414) 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 -- 172.21.15.144:0/4168800844 learned_addr learned my addr 172.21.15.144:0/4168800844 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ecb03700 1 --2- 172.21.15.144:0/4168800844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e80cbdf0 0x7f94e806b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 -- 172.21.15.144:0/4168800844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e80cbdf0 msgr2=0x7f94e806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 --2- 172.21.15.144:0/4168800844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e80cbdf0 0x7f94e806b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 -- 172.21.15.144:0/4168800844 >> v1:172.21.15.177:6789/0 conn(0x7f94e806b8c0 legacy=0x7f94e8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.204+0000 7f94ed304700 1 -- 172.21.15.144:0/4168800844 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e815d750 con 0x7f94e815cef0 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ed304700 1 --2- 172.21.15.144:0/4168800844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e815cef0 0x7f94e80af400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f94d80098f0 tx=0x7f94d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f92a0fdc8d6246 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94dffff700 1 -- 172.21.15.144:0/4168800844 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8011830 con 0x7f94e815cef0 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94dffff700 1 -- 172.21.15.144:0/4168800844 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8011990 con 0x7f94e815cef0 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94dffff700 1 -- 172.21.15.144:0/4168800844 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8011d20 con 0x7f94e815cef0 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 -- 172.21.15.144:0/4168800844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e815cef0 msgr2=0x7f94e80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 --2- 172.21.15.144:0/4168800844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e815cef0 0x7f94e80af400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f94d80098f0 tx=0x7f94d8009c00 comp rx=0 tx=0).stop 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 -- 172.21.15.144:0/4168800844 shutdown_connections 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 --2- 172.21.15.144:0/4168800844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e815cef0 0x7f94e80af400 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 --2- 172.21.15.144:0/4168800844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e80cbdf0 0x7f94e806b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 -- 172.21.15.144:0/4168800844 >> 172.21.15.144:0/4168800844 conn(0x7f94e80d0000 msgr2=0x7f94e80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:54.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.205+0000 7f94ef568700 1 -- 172.21.15.144:0/4168800844 shutdown_connections 2023-11-28T11:53:54.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.206+0000 7f94ef568700 1 -- 172.21.15.144:0/4168800844 wait complete. 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 Processor -- start 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 -- start start 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e806b8c0 0x7f94e80c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 0x7f94e80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94e815cef0 0x7f94e815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e81e40f0 con 0x7f94e80cbdf0 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80c77b0 con 0x7f94e815cef0 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.207+0000 7f94ef568700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80c7a90 con 0x7f94e806b8c0 2023-11-28T11:53:54.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 0x7f94e80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 0x7f94e80c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38422/0 (socket says 172.21.15.144:38422) 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 -- 172.21.15.144:0/3930501925 learned_addr learned my addr 172.21.15.144:0/3930501925 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ed304700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e806b8c0 0x7f94e80c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94edb05700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94e815cef0 0x7f94e815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 -- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e806b8c0 msgr2=0x7f94e80c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e806b8c0 0x7f94e80c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 -- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94e815cef0 msgr2=0x7f94e815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94e815cef0 0x7f94e815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 -- 172.21.15.144:0/3930501925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8009550 con 0x7f94e80cbdf0 2023-11-28T11:53:54.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ecb03700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 0x7f94e80c70d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e9b0 tx=0x7f94e400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4009710 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400ee50 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e815dd10 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400ca50 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.208+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e815da30 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.209+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e815dff0 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.210+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94e4019030 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.210+0000 7f94ddffb700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94cc0682f0 0x7f94cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.210+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94e4085c20 con 0x7f94e80cbdf0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.210+0000 7f94ed304700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94cc0682f0 0x7f94cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:54.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.210+0000 7f94ed304700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94cc0682f0 0x7f94cc06a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f94d8014040 tx=0x7f94d8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:54.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.214+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94e4052390 con 0x7f94e80cbdf0 2023-11-28T11:53:54.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:54.447+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f94e81e3e50 con 0x7f94e80cbdf0 2023-11-28T11:53:55.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.345+0000 7f94ddffb700 1 -- 172.21.15.144:0/3930501925 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f94e81e3e50 con 0x7f94e80cbdf0 2023-11-28T11:53:55.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.347+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94cc0682f0 msgr2=0x7f94cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94cc0682f0 0x7f94cc06a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f94d8014040 tx=0x7f94d8008030 comp rx=0 tx=0).stop 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 msgr2=0x7f94e80c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 0x7f94e80c70d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f94e400e9b0 tx=0x7f94e400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 shutdown_connections 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94cc0682f0 0x7f94cc06a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94e815cef0 0x7f94e815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94e806b8c0 0x7f94e80c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 --2- 172.21.15.144:0/3930501925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e80cbdf0 0x7f94e80c70d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 >> 172.21.15.144:0/3930501925 conn(0x7f94e80d0000 msgr2=0x7f94e80cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:55.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 shutdown_connections 2023-11-28T11:53:55.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.348+0000 7f94ef568700 1 -- 172.21.15.144:0/3930501925 wait complete. 2023-11-28T11:53:55.364 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:53:55.365 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:53:55.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.492+0000 7fb4054a5700 1 Processor -- start 2023-11-28T11:53:55.494 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.492+0000 7fb4054a5700 1 -- start start 2023-11-28T11:53:55.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.493+0000 7fb4054a5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:55.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.493+0000 7fb4054a5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb400159860 0x7fb4000b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:55.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.493+0000 7fb4054a5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4000ccc00 con 0x7fb4000cdc80 2023-11-28T11:53:55.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.493+0000 7fb4054a5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4000ccec0 con 0x7fb400159860 2023-11-28T11:53:55.497 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.493+0000 7fb4054a5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000cd1a0 con 0x7fb400159490 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3feffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb400159860 0x7fb4000b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3feffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:50510/0 (socket says 172.21.15.144:50510) 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3ff7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fb4000cdc80 0x7fb4000c94c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55634/0 (socket says 172.21.15.144:55634) 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3feffd700 1 -- 172.21.15.144:0/3370526867 learned_addr learned my addr 172.21.15.144:0/3370526867 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb400159860 0x7fb4000b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:40704/0 (socket says 172.21.15.144:40704) 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 -- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 msgr2=0x7fb4000b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 --2- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 -- 172.21.15.144:0/3370526867 >> v1:172.21.15.177:6789/0 conn(0x7fb4000cdc80 legacy=0x7fb4000c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fdffb700 1 -- 172.21.15.144:0/3370526867 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597988114 0 0) 0x7fb4000ccc00 con 0x7fb4000cdc80 2023-11-28T11:53:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 -- 172.21.15.144:0/3370526867 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000cd480 con 0x7fb400159860 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fe7fc700 1 --2- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb400159860 0x7fb4000b0bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400bae0 tx=0x7fb3f400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b59cd81bc1ef93a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fdffb700 1 -- 172.21.15.144:0/3370526867 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400dc70 con 0x7fb400159860 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fdffb700 1 -- 172.21.15.144:0/3370526867 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f400ddd0 con 0x7fb400159860 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.494+0000 7fb3fdffb700 1 -- 172.21.15.144:0/3370526867 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4010690 con 0x7fb400159860 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 -- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb400159860 msgr2=0x7fb4000b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 --2- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb400159860 0x7fb4000b0bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400bae0 tx=0x7fb3f400bdf0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 -- 172.21.15.144:0/3370526867 shutdown_connections 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 --2- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 --2- 172.21.15.144:0/3370526867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb400159860 0x7fb4000b0bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 -- 172.21.15.144:0/3370526867 >> 172.21.15.144:0/3370526867 conn(0x7fb4000cb670 msgr2=0x7fb4000cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:55.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 -- 172.21.15.144:0/3370526867 shutdown_connections 2023-11-28T11:53:55.500 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.495+0000 7fb4054a5700 1 -- 172.21.15.144:0/3370526867 wait complete. 2023-11-28T11:53:55.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.497+0000 7fb4054a5700 1 Processor -- start 2023-11-28T11:53:55.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.497+0000 7fb4054a5700 1 -- start start 2023-11-28T11:53:55.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4000cdc80 0x7fb400152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:55.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:55.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 0x7fb400152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:55.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4001528c0 con 0x7fb4000c5c30 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000c6130 con 0x7fb4000cdc80 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000c6410 con 0x7fb400159490 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 0x7fb400152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 0x7fb400152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38438/0 (socket says 172.21.15.144:38438) 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 -- 172.21.15.144:0/629096774 learned_addr learned my addr 172.21.15.144:0/629096774 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3feffd700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4000cdc80 0x7fb400152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3fe7fc700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 -- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 msgr2=0x7fb4000b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:55.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 -- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4000cdc80 msgr2=0x7fb400152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4000cdc80 0x7fb400152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 -- 172.21.15.144:0/629096774 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f400b740 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3feffd700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4000cdc80 0x7fb400152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3ff7fe700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 0x7fb400152ce0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8002230 tx=0x7fb3f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800ac20 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f800ada0 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4000c9bd0 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8007310 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.498+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4000cd760 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.499+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4000c9d60 con 0x7fb4000c5c30 2023-11-28T11:53:55.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.500+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb3f8010070 con 0x7fb4000c5c30 2023-11-28T11:53:55.506 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.501+0000 7fb3e77fe700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb3ec0682f0 0x7fb3ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:55.506 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.501+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb3f8085ad0 con 0x7fb4000c5c30 2023-11-28T11:53:55.506 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.501+0000 7fb3feffd700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb3ec0682f0 0x7fb3ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:55.506 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.501+0000 7fb3feffd700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb3ec0682f0 0x7fb3ec06a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009c20 tx=0x7fb3f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:55.506 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.505+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3f80522d0 con 0x7fb4000c5c30 2023-11-28T11:53:55.735 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:55.733+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb400152a00 con 0x7fb4000c5c30 2023-11-28T11:53:56.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.354+0000 7fb3e77fe700 1 -- 172.21.15.144:0/629096774 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb400152a00 con 0x7fb4000c5c30 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.356+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb3ec0682f0 msgr2=0x7fb3ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.356+0000 7fb4054a5700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb3ec0682f0 0x7fb3ec06a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009c20 tx=0x7fb3f0008040 comp rx=0 tx=0).stop 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.356+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 msgr2=0x7fb400152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.356+0000 7fb4054a5700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 0x7fb400152ce0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8002230 tx=0x7fb3f800a060 comp rx=0 tx=0).stop 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 shutdown_connections 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4000c5c30 0x7fb400152ce0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb3ec0682f0 0x7fb3ec06a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4000cdc80 0x7fb400152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.359 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 --2- 172.21.15.144:0/629096774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb400159490 0x7fb4000b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 >> 172.21.15.144:0/629096774 conn(0x7fb4000cb670 msgr2=0x7fb400157100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:56.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 shutdown_connections 2023-11-28T11:53:56.360 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.357+0000 7fb4054a5700 1 -- 172.21.15.144:0/629096774 wait complete. 2023-11-28T11:53:56.373 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:53:56.374 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:56.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.497+0000 7fe55c097700 1 Processor -- start 2023-11-28T11:53:56.499 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.498+0000 7fe55c097700 1 -- start start 2023-11-28T11:53:56.502 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.498+0000 7fe55c097700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5540cb0e0 0x7fe554065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.502 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.498+0000 7fe55c097700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe55406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.498+0000 7fe55c097700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe554066580 con 0x7fe55406ec90 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.498+0000 7fe55c097700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5540cded0 con 0x7fe5540cb0e0 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.498+0000 7fe55c097700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540ce1b0 con 0x7fe5540cec70 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559632700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5540cb0e0 0x7fe554065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559e33700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe55406ec90 0x7fe5540ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55154/0 (socket says 172.21.15.144:55154) 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559e33700 1 -- 172.21.15.144:0/671897493 learned_addr learned my addr 172.21.15.144:0/671897493 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe55a634700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe55406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559632700 1 -- 172.21.15.144:0/671897493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 msgr2=0x7fe55406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559632700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe55406d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559632700 1 -- 172.21.15.144:0/671897493 >> v1:172.21.15.144:6789/0 conn(0x7fe55406ec90 legacy=0x7fe5540ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.503 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559632700 1 -- 172.21.15.144:0/671897493 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540ce490 con 0x7fe5540cb0e0 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe558e31700 1 -- 172.21.15.144:0/671897493 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43862862 0 0) 0x7fe554066580 con 0x7fe55406ec90 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe55a634700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe55406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe559632700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5540cb0e0 0x7fe554065ef0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe550009840 tx=0x7fe550009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afbb54eba8b4a2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe558e31700 1 -- 172.21.15.144:0/671897493 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550011830 con 0x7fe5540cb0e0 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe558e31700 1 -- 172.21.15.144:0/671897493 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe550011990 con 0x7fe5540cb0e0 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.499+0000 7fe558e31700 1 -- 172.21.15.144:0/671897493 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550011d20 con 0x7fe5540cb0e0 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 -- 172.21.15.144:0/671897493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5540cb0e0 msgr2=0x7fe554065ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5540cb0e0 0x7fe554065ef0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe550009840 tx=0x7fe550009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 -- 172.21.15.144:0/671897493 shutdown_connections 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe55406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 --2- 172.21.15.144:0/671897493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5540cb0e0 0x7fe554065ef0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.504 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 -- 172.21.15.144:0/671897493 >> 172.21.15.144:0/671897493 conn(0x7fe5540b12b0 msgr2=0x7fe5540b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:56.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 -- 172.21.15.144:0/671897493 shutdown_connections 2023-11-28T11:53:56.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.500+0000 7fe55c097700 1 -- 172.21.15.144:0/671897493 wait complete. 2023-11-28T11:53:56.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.501+0000 7fe55c097700 1 Processor -- start 2023-11-28T11:53:56.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.501+0000 7fe55c097700 1 -- start start 2023-11-28T11:53:56.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55c097700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 0x7fe5541570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55c097700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5540cb0e0 0x7fe554070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55c097700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe554070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55c097700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe554159a20 con 0x7fe55406ec90 2023-11-28T11:53:56.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55c097700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe554159d00 con 0x7fe5540cb0e0 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55c097700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe554159fe0 con 0x7fe5540cec70 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 0x7fe5541570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 0x7fe5541570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38448/0 (socket says 172.21.15.144:38448) 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 -- 172.21.15.144:0/176266261 learned_addr learned my addr 172.21.15.144:0/176266261 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 -- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 msgr2=0x7fe554070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55a634700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe554070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559632700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5540cb0e0 0x7fe554070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe554070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 -- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5540cb0e0 msgr2=0x7fe554070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5540cb0e0 0x7fe554070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559e33700 1 -- 172.21.15.144:0/176266261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5500094a0 con 0x7fe55406ec90 2023-11-28T11:53:56.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe559632700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5540cb0e0 0x7fe554070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.502+0000 7fe55a634700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe554070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.503+0000 7fe559e33700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 0x7fe5541570c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c550 tx=0x7fe54800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.503+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548004c80 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.503+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548004e00 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.503+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5540cb6c0 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.503+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548007310 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.503+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5540ce770 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.504+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55406f060 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.504+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe54801a020 con 0x7fe55406ec90 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.504+0000 7fe5467fc700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5380682f0 0x7fe53806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.505+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5480858e0 con 0x7fe55406ec90 2023-11-28T11:53:56.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.505+0000 7fe559632700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5380682f0 0x7fe53806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.505+0000 7fe559632700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5380682f0 0x7fe53806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe554072180 tx=0x7fe550008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:56.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.509+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5480521f0 con 0x7fe55406ec90 2023-11-28T11:53:56.737 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.736+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5540cf110 con 0x7fe55406ec90 2023-11-28T11:53:56.738 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.736+0000 7fe5467fc700 1 -- 172.21.15.144:0/176266261 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fe548055810 con 0x7fe55406ec90 2023-11-28T11:53:56.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:56.739 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:53:56.350260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:53:56.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5380682f0 msgr2=0x7fe53806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5380682f0 0x7fe53806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe554072180 tx=0x7fe550008030 comp rx=0 tx=0).stop 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 msgr2=0x7fe5541570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 0x7fe5541570c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c550 tx=0x7fe54800c910 comp rx=0 tx=0).stop 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 shutdown_connections 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe55406ec90 0x7fe5541570c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5380682f0 0x7fe53806a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5540cec70 0x7fe554070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 --2- 172.21.15.144:0/176266261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5540cb0e0 0x7fe554070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 >> 172.21.15.144:0/176266261 conn(0x7fe5540b12b0 msgr2=0x7fe5540b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:56.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 shutdown_connections 2023-11-28T11:53:56.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.738+0000 7fe55c097700 1 -- 172.21.15.144:0/176266261 wait complete. 2023-11-28T11:53:56.742 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 31 2023-11-28T11:53:56.752 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:53:56.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.871+0000 7f9e8d8bf700 1 Processor -- start 2023-11-28T11:53:56.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.871+0000 7f9e8d8bf700 1 -- start start 2023-11-28T11:53:56.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e8d8bf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9e88071040 0x7f9e88071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e8d8bf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806dee0 0x7f9e8806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e8d8bf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e8806cdc0 0x7f9e8806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e8d8bf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e88071950 con 0x7f9e8806cdc0 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e8d8bf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8815d060 con 0x7f9e88071040 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e8d8bf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8815d340 con 0x7f9e8806dee0 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e8806cdc0 0x7f9e8806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e8806cdc0 0x7f9e8806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38460/0 (socket says 172.21.15.144:38460) 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 -- 172.21.15.144:0/4112378516 learned_addr learned my addr 172.21.15.144:0/4112378516 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e867fc700 1 --2- 172.21.15.144:0/4112378516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806dee0 0x7f9e8806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e86ffd700 1 --2- 172.21.15.144:0/4112378516 >> v2:172.21.15.177:3300/0 conn(0x7f9e88071040 0x7f9e88071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 -- 172.21.15.144:0/4112378516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806dee0 msgr2=0x7f9e8806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 --2- 172.21.15.144:0/4112378516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806dee0 0x7f9e8806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 -- 172.21.15.144:0/4112378516 >> v2:172.21.15.177:3300/0 conn(0x7f9e88071040 msgr2=0x7f9e88071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 --2- 172.21.15.144:0/4112378516 >> v2:172.21.15.177:3300/0 conn(0x7f9e88071040 0x7f9e88071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 -- 172.21.15.144:0/4112378516 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8815d620 con 0x7f9e8806cdc0 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.872+0000 7f9e877fe700 1 --2- 172.21.15.144:0/4112378516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8806cdc0 0x7f9e8806d240 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9e880d0f50 tx=0x7f9e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806f5bb39c1fabac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e85ffb700 1 -- 172.21.15.144:0/4112378516 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000e070 con 0x7f9e8806cdc0 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e85ffb700 1 -- 172.21.15.144:0/4112378516 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000b910 con 0x7f9e8806cdc0 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e85ffb700 1 -- 172.21.15.144:0/4112378516 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000bcc0 con 0x7f9e8806cdc0 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/4112378516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8806cdc0 msgr2=0x7f9e8806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/4112378516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8806cdc0 0x7f9e8806d240 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9e880d0f50 tx=0x7f9e8000b330 comp rx=0 tx=0).stop 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/4112378516 shutdown_connections 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/4112378516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e8806cdc0 0x7f9e8806d240 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/4112378516 >> v2:172.21.15.177:3300/0 conn(0x7f9e88071040 0x7f9e88071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/4112378516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806dee0 0x7f9e8806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/4112378516 >> 172.21.15.144:0/4112378516 conn(0x7f9e880d0000 msgr2=0x7f9e880d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:56.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/4112378516 shutdown_connections 2023-11-28T11:53:56.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.873+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/4112378516 wait complete. 2023-11-28T11:53:56.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.874+0000 7f9e8d8bf700 1 Processor -- start 2023-11-28T11:53:56.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 -- start start 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806cdc0 0x7f9e881570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8806dee0 0x7f9e880c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 0x7f9e880b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88157600 con 0x7f9e88071040 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88159850 con 0x7f9e8806dee0 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e8d8bf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880c55a0 con 0x7f9e8806cdc0 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 0x7f9e880b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 0x7f9e880b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38466/0 (socket says 172.21.15.144:38466) 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.875+0000 7f9e877fe700 1 -- 172.21.15.144:0/3323641809 learned_addr learned my addr 172.21.15.144:0/3323641809 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e867fc700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8806dee0 0x7f9e880c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e86ffd700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806cdc0 0x7f9e881570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e877fe700 1 -- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806cdc0 msgr2=0x7f9e881570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e877fe700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806cdc0 0x7f9e881570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e877fe700 1 -- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8806dee0 msgr2=0x7f9e880c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e877fe700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8806dee0 0x7f9e880c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e877fe700 1 -- 172.21.15.144:0/3323641809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80009b30 con 0x7f9e88071040 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e86ffd700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806cdc0 0x7f9e881570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e867fc700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8806dee0 0x7f9e880c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e877fe700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 0x7f9e880b4960 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9e8806c7b0 tx=0x7f9e80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000ce80 con 0x7f9e88071040 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80004250 con 0x7f9e88071040 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80004600 con 0x7f9e88071040 2023-11-28T11:53:56.882 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8806e870 con 0x7f9e88071040 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.876+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8815d900 con 0x7f9e88071040 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.877+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8815ccd0 con 0x7f9e88071040 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.877+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e8002b070 con 0x7f9e88071040 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.878+0000 7f9e6ffff700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e740682f0 0x7f9e7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.878+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e80086f00 con 0x7f9e88071040 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.878+0000 7f9e86ffd700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e740682f0 0x7f9e7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.878+0000 7f9e86ffd700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e740682f0 0x7f9e7406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009b90 tx=0x7f9e7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:56.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:56.882+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e80053760 con 0x7f9e88071040 2023-11-28T11:53:57.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.066+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e880b18f0 con 0x7f9e88071040 2023-11-28T11:53:57.069 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.067+0000 7f9e6ffff700 1 -- 172.21.15.144:0/3323641809 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17911 (secure 0 0 0) 0x7f9e8001f020 con 0x7f9e88071040 2023-11-28T11:53:57.069 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:57.069 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":60,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:53:54.339507+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:53:57.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e740682f0 msgr2=0x7f9e7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.071 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e740682f0 0x7f9e7406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009b90 tx=0x7f9e7c008040 comp rx=0 tx=0).stop 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 msgr2=0x7f9e880b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 0x7f9e880b4960 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9e8806c7b0 tx=0x7f9e80004ad0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 shutdown_connections 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88071040 0x7f9e880b4960 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9e740682f0 0x7f9e7406a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e8806cdc0 0x7f9e881570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 --2- 172.21.15.144:0/3323641809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e8806dee0 0x7f9e880c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 >> 172.21.15.144:0/3323641809 conn(0x7f9e880d0000 msgr2=0x7f9e8815ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.069+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 shutdown_connections 2023-11-28T11:53:57.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.070+0000 7f9e8d8bf700 1 -- 172.21.15.144:0/3323641809 wait complete. 2023-11-28T11:53:57.084 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:53:57.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:53:57.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.202+0000 7ff60bab3700 1 Processor -- start 2023-11-28T11:53:57.204 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.202+0000 7ff60bab3700 1 -- start start 2023-11-28T11:53:57.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.203+0000 7ff60bab3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.203+0000 7ff60bab3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60406bbf0 con 0x7ff604160740 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.203+0000 7ff60bab3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6040c9e10 con 0x7ff60406b060 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.203+0000 7ff60bab3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040ca720 con 0x7ff604160b10 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.203+0000 7ff60984f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff604160740 0x7ff6040cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55182/0 (socket says 172.21.15.144:55182) 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.203+0000 7ff60984f700 1 -- 172.21.15.144:0/1509309423 learned_addr learned my addr 172.21.15.144:0/1509309423 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60904e700 1 --2- 172.21.15.144:0/1509309423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892848499 0 0) 0x7ff60406bbf0 con 0x7ff604160740 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f00035b0 con 0x7ff604160740 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791008141 0 0) 0x7ff6040c9e10 con 0x7ff60406b060 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60406bbf0 con 0x7ff60406b060 2023-11-28T11:53:57.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60904e700 1 -- 172.21.15.144:0/1509309423 >> v1:172.21.15.177:6789/0 conn(0x7ff60406b060 legacy=0x7ff60406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60904e700 1 -- 172.21.15.144:0/1509309423 >> v1:172.21.15.144:6789/0 conn(0x7ff604160740 legacy=0x7ff6040cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60904e700 1 -- 172.21.15.144:0/1509309423 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60406efc0 con 0x7ff604160b10 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60904e700 1 --2- 172.21.15.144:0/1509309423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040cd390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009870 tx=0x7ff5f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353e7f75df4af406 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4011830 con 0x7ff604160b10 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4011990 con 0x7ff604160b10 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.204+0000 7ff60884d700 1 -- 172.21.15.144:0/1509309423 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4011d20 con 0x7ff604160b10 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 -- 172.21.15.144:0/1509309423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 msgr2=0x7ff6040cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1509309423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040cd390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009870 tx=0x7ff5f4009b80 comp rx=0 tx=0).stop 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 -- 172.21.15.144:0/1509309423 shutdown_connections 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1509309423 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040cd390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 -- 172.21.15.144:0/1509309423 >> 172.21.15.144:0/1509309423 conn(0x7ff6040b12b0 msgr2=0x7ff6040b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 -- 172.21.15.144:0/1509309423 shutdown_connections 2023-11-28T11:53:57.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.205+0000 7ff60bab3700 1 -- 172.21.15.144:0/1509309423 wait complete. 2023-11-28T11:53:57.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.206+0000 7ff60bab3700 1 Processor -- start 2023-11-28T11:53:57.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.206+0000 7ff60bab3700 1 -- start start 2023-11-28T11:53:57.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60bab3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff60406b060 0x7ff604157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60bab3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 0x7ff6040c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60bab3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60bab3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60406c7b0 con 0x7ff604160740 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60bab3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c8ca0 con 0x7ff60406b060 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60bab3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c8f80 con 0x7ff604160b10 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 0x7ff6040c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 0x7ff6040c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38474/0 (socket says 172.21.15.144:38474) 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 -- 172.21.15.144:0/1269374861 learned_addr learned my addr 172.21.15.144:0/1269374861 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 -- 172.21.15.144:0/1269374861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 msgr2=0x7ff6040c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 -- 172.21.15.144:0/1269374861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff60406b060 msgr2=0x7ff604157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff60406b060 0x7ff604157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 -- 172.21.15.144:0/1269374861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40094a0 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff60904e700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 0x7ff6040c80e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40090b0 tx=0x7ff5f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.207+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f401a860 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.208+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f401a9c0 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.208+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6040ca3b0 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.208+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f401ad30 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.208+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6040ca0d0 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.208+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6040cd8d0 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.209+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff5f402b030 con 0x7ff604160740 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.209+0000 7ff5fa7fc700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff5e80682f0 0x7ff5e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.209+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5f4086f50 con 0x7ff604160740 2023-11-28T11:53:57.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.209+0000 7ff60984f700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff5e80682f0 0x7ff5e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.210+0000 7ff60984f700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff5e80682f0 0x7ff5e806a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff600005910 tx=0x7ff6000079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.214+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff5f40b85e0 con 0x7ff604160740 2023-11-28T11:53:57.395 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.393+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff604072960 con 0x7ff5e80682f0 2023-11-28T11:53:57.403 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.402+0000 7ff5fa7fc700 1 -- 172.21.15.144:0/1269374861 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311897 (secure 0 0 0) 0x7ff604072960 con 0x7ff5e80682f0 2023-11-28T11:53:57.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff5e80682f0 msgr2=0x7ff5e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff5e80682f0 0x7ff5e806a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff600005910 tx=0x7ff6000079e0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 msgr2=0x7ff6040c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 0x7ff6040c80e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40090b0 tx=0x7ff5f4004750 comp rx=0 tx=0).stop 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 shutdown_connections 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff5e80682f0 0x7ff5e806a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff604160b10 0x7ff6040c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff60406b060 0x7ff604157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.411 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 --2- 172.21.15.144:0/1269374861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff604160740 0x7ff6040c80e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 >> 172.21.15.144:0/1269374861 conn(0x7ff6040b12b0 msgr2=0x7ff6040acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:57.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 shutdown_connections 2023-11-28T11:53:57.412 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.405+0000 7ff60bab3700 1 -- 172.21.15.144:0/1269374861 wait complete. 2023-11-28T11:53:57.412 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:53:57.419 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2023-11-28T11:53:56.517880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81996,"kb_used_data":6732,"kb_used_omap":0,"kb_used_meta":75200,"kb_avail":754892724,"statfs":{"total":773094113280,"available":773010149376,"internally_reserved":0,"allocated":6893568,"data_stored":3649157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77004800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.591716"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365099+0000","last_change":"2023-11-28T11:53:53.355794+0000","last_active":"2023-11-28T11:53:53.365099+0000","last_peered":"2023-11-28T11:53:53.365099+0000","last_clean":"2023-11-28T11:53:53.365099+0000","last_became_active":"2023-11-28T11:53:53.355336+0000","last_became_peered":"2023-11-28T11:53:53.355336+0000","last_unstale":"2023-11-28T11:53:53.365099+0000","last_undegraded":"2023-11-28T11:53:53.365099+0000","last_fullsized":"2023-11-28T11:53:53.365099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:19.117644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367391+0000","last_change":"2023-11-28T11:53:53.364706+0000","last_active":"2023-11-28T11:53:53.367391+0000","last_peered":"2023-11-28T11:53:53.367391+0000","last_clean":"2023-11-28T11:53:53.367391+0000","last_became_active":"2023-11-28T11:53:53.364111+0000","last_became_peered":"2023-11-28T11:53:53.364111+0000","last_unstale":"2023-11-28T11:53:53.367391+0000","last_undegraded":"2023-11-28T11:53:53.367391+0000","last_fullsized":"2023-11-28T11:53:53.367391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:58.872124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366648+0000","last_change":"2023-11-28T11:53:53.364232+0000","last_active":"2023-11-28T11:53:53.366648+0000","last_peered":"2023-11-28T11:53:53.366648+0000","last_clean":"2023-11-28T11:53:53.366648+0000","last_became_active":"2023-11-28T11:53:53.363973+0000","last_became_peered":"2023-11-28T11:53:53.363973+0000","last_unstale":"2023-11-28T11:53:53.366648+0000","last_undegraded":"2023-11-28T11:53:53.366648+0000","last_fullsized":"2023-11-28T11:53:53.366648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:27.701979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348854+0000","last_change":"2023-11-28T11:53:53.364319+0000","last_active":"2023-11-28T11:53:54.348854+0000","last_peered":"2023-11-28T11:53:54.348854+0000","last_clean":"2023-11-28T11:53:54.348854+0000","last_became_active":"2023-11-28T11:53:53.363933+0000","last_became_peered":"2023-11-28T11:53:53.363933+0000","last_unstale":"2023-11-28T11:53:54.348854+0000","last_undegraded":"2023-11-28T11:53:54.348854+0000","last_fullsized":"2023-11-28T11:53:54.348854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:27.701979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.371686+0000","last_change":"2023-11-28T11:53:53.368196+0000","last_active":"2023-11-28T11:53:53.371686+0000","last_peered":"2023-11-28T11:53:53.371686+0000","last_clean":"2023-11-28T11:53:53.371686+0000","last_became_active":"2023-11-28T11:53:53.367915+0000","last_became_peered":"2023-11-28T11:53:53.367915+0000","last_unstale":"2023-11-28T11:53:53.371686+0000","last_undegraded":"2023-11-28T11:53:53.371686+0000","last_fullsized":"2023-11-28T11:53:53.371686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:19.117644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823332+0000","last_change":"2023-11-28T11:53:53.366020+0000","last_active":"2023-11-28T11:53:54.823332+0000","last_peered":"2023-11-28T11:53:54.823332+0000","last_clean":"2023-11-28T11:53:54.823332+0000","last_became_active":"2023-11-28T11:53:53.365791+0000","last_became_peered":"2023-11-28T11:53:53.365791+0000","last_unstale":"2023-11-28T11:53:54.823332+0000","last_undegraded":"2023-11-28T11:53:54.823332+0000","last_fullsized":"2023-11-28T11:53:54.823332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:33.277898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.362502+0000","last_change":"2023-11-28T11:53:53.360962+0000","last_active":"2023-11-28T11:53:53.362502+0000","last_peered":"2023-11-28T11:53:53.362502+0000","last_clean":"2023-11-28T11:53:53.362502+0000","last_became_active":"2023-11-28T11:53:53.360718+0000","last_became_peered":"2023-11-28T11:53:53.360718+0000","last_unstale":"2023-11-28T11:53:53.362502+0000","last_undegraded":"2023-11-28T11:53:53.362502+0000","last_fullsized":"2023-11-28T11:53:53.362502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:41.319939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347331+0000","last_change":"2023-11-28T11:53:53.358618+0000","last_active":"2023-11-28T11:53:54.347331+0000","last_peered":"2023-11-28T11:53:54.347331+0000","last_clean":"2023-11-28T11:53:54.347331+0000","last_became_active":"2023-11-28T11:53:53.358351+0000","last_became_peered":"2023-11-28T11:53:53.358351+0000","last_unstale":"2023-11-28T11:53:54.347331+0000","last_undegraded":"2023-11-28T11:53:54.347331+0000","last_fullsized":"2023-11-28T11:53:54.347331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:57.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366694+0000","last_change":"2023-11-28T11:53:53.365311+0000","last_active":"2023-11-28T11:53:53.366694+0000","last_peered":"2023-11-28T11:53:53.366694+0000","last_clean":"2023-11-28T11:53:53.366694+0000","last_became_active":"2023-11-28T11:53:53.364487+0000","last_became_peered":"2023-11-28T11:53:53.364487+0000","last_unstale":"2023-11-28T11:53:53.366694+0000","last_undegraded":"2023-11-28T11:53:53.366694+0000","last_fullsized":"2023-11-28T11:53:53.366694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:17:18.508153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.360514+0000","last_change":"2023-11-28T11:53:53.357892+0000","last_active":"2023-11-28T11:53:53.360514+0000","last_peered":"2023-11-28T11:53:53.360514+0000","last_clean":"2023-11-28T11:53:53.360514+0000","last_became_active":"2023-11-28T11:53:53.356964+0000","last_became_peered":"2023-11-28T11:53:53.356964+0000","last_unstale":"2023-11-28T11:53:53.360514+0000","last_undegraded":"2023-11-28T11:53:53.360514+0000","last_fullsized":"2023-11-28T11:53:53.360514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:30.442863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348510+0000","last_change":"2023-11-28T11:53:53.365848+0000","last_active":"2023-11-28T11:53:54.348510+0000","last_peered":"2023-11-28T11:53:54.348510+0000","last_clean":"2023-11-28T11:53:54.348510+0000","last_became_active":"2023-11-28T11:53:53.365045+0000","last_became_peered":"2023-11-28T11:53:53.365045+0000","last_unstale":"2023-11-28T11:53:54.348510+0000","last_undegraded":"2023-11-28T11:53:54.348510+0000","last_fullsized":"2023-11-28T11:53:54.348510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:18.360820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347448+0000","last_change":"2023-11-28T11:53:53.358033+0000","last_active":"2023-11-28T11:53:54.347448+0000","last_peered":"2023-11-28T11:53:54.347448+0000","last_clean":"2023-11-28T11:53:54.347448+0000","last_became_active":"2023-11-28T11:53:53.357727+0000","last_became_peered":"2023-11-28T11:53:53.357727+0000","last_unstale":"2023-11-28T11:53:54.347448+0000","last_undegraded":"2023-11-28T11:53:54.347448+0000","last_fullsized":"2023-11-28T11:53:54.347448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:42.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347520+0000","last_change":"2023-11-28T11:53:53.367259+0000","last_active":"2023-11-28T11:53:54.347520+0000","last_peered":"2023-11-28T11:53:54.347520+0000","last_clean":"2023-11-28T11:53:54.347520+0000","last_became_active":"2023-11-28T11:53:53.366982+0000","last_became_peered":"2023-11-28T11:53:53.366982+0000","last_unstale":"2023-11-28T11:53:54.347520+0000","last_undegraded":"2023-11-28T11:53:54.347520+0000","last_fullsized":"2023-11-28T11:53:54.347520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:28.183193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366084+0000","last_change":"2023-11-28T11:53:53.361963+0000","last_active":"2023-11-28T11:53:53.366084+0000","last_peered":"2023-11-28T11:53:53.366084+0000","last_clean":"2023-11-28T11:53:53.366084+0000","last_became_active":"2023-11-28T11:53:53.361656+0000","last_became_peered":"2023-11-28T11:53:53.361656+0000","last_unstale":"2023-11-28T11:53:53.366084+0000","last_undegraded":"2023-11-28T11:53:53.366084+0000","last_fullsized":"2023-11-28T11:53:53.366084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:20.156694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363018+0000","last_change":"2023-11-28T11:53:53.359500+0000","last_active":"2023-11-28T11:53:53.363018+0000","last_peered":"2023-11-28T11:53:53.363018+0000","last_clean":"2023-11-28T11:53:53.363018+0000","last_became_active":"2023-11-28T11:53:53.354835+0000","last_became_peered":"2023-11-28T11:53:53.354835+0000","last_unstale":"2023-11-28T11:53:53.363018+0000","last_undegraded":"2023-11-28T11:53:53.363018+0000","last_fullsized":"2023-11-28T11:53:53.363018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:42.420815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363619+0000","last_change":"2023-11-28T11:53:53.352510+0000","last_active":"2023-11-28T11:53:53.363619+0000","last_peered":"2023-11-28T11:53:53.363619+0000","last_clean":"2023-11-28T11:53:53.363619+0000","last_became_active":"2023-11-28T11:53:53.352277+0000","last_became_peered":"2023-11-28T11:53:53.352277+0000","last_unstale":"2023-11-28T11:53:53.363619+0000","last_undegraded":"2023-11-28T11:53:53.363619+0000","last_fullsized":"2023-11-28T11:53:53.363619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:59.032830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.369524+0000","last_change":"2023-11-28T11:53:53.368953+0000","last_active":"2023-11-28T11:53:53.369524+0000","last_peered":"2023-11-28T11:53:53.369524+0000","last_clean":"2023-11-28T11:53:53.369524+0000","last_became_active":"2023-11-28T11:53:53.368752+0000","last_became_peered":"2023-11-28T11:53:53.368752+0000","last_unstale":"2023-11-28T11:53:53.369524+0000","last_undegraded":"2023-11-28T11:53:53.369524+0000","last_fullsized":"2023-11-28T11:53:53.369524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:06.472324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348524+0000","last_change":"2023-11-28T11:53:53.363865+0000","last_active":"2023-11-28T11:53:54.348524+0000","last_peered":"2023-11-28T11:53:54.348524+0000","last_clean":"2023-11-28T11:53:54.348524+0000","last_became_active":"2023-11-28T11:53:53.363551+0000","last_became_peered":"2023-11-28T11:53:53.363551+0000","last_unstale":"2023-11-28T11:53:54.348524+0000","last_undegraded":"2023-11-28T11:53:54.348524+0000","last_fullsized":"2023-11-28T11:53:54.348524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:37.461966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357711+0000","last_change":"2023-11-28T11:53:53.352764+0000","last_active":"2023-11-28T11:53:53.357711+0000","last_peered":"2023-11-28T11:53:53.357711+0000","last_clean":"2023-11-28T11:53:53.357711+0000","last_became_active":"2023-11-28T11:53:53.352491+0000","last_became_peered":"2023-11-28T11:53:53.352491+0000","last_unstale":"2023-11-28T11:53:53.357711+0000","last_undegraded":"2023-11-28T11:53:53.357711+0000","last_fullsized":"2023-11-28T11:53:53.357711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:58.872124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347708+0000","last_change":"2023-11-28T11:53:53.349348+0000","last_active":"2023-11-28T11:53:54.347708+0000","last_peered":"2023-11-28T11:53:54.347708+0000","last_clean":"2023-11-28T11:53:54.347708+0000","last_became_active":"2023-11-28T11:53:53.349086+0000","last_became_peered":"2023-11-28T11:53:53.349086+0000","last_unstale":"2023-11-28T11:53:54.347708+0000","last_undegraded":"2023-11-28T11:53:54.347708+0000","last_fullsized":"2023-11-28T11:53:54.347708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:12.565467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347631+0000","last_change":"2023-11-28T11:53:53.355325+0000","last_active":"2023-11-28T11:53:54.347631+0000","last_peered":"2023-11-28T11:53:54.347631+0000","last_clean":"2023-11-28T11:53:54.347631+0000","last_became_active":"2023-11-28T11:53:53.355066+0000","last_became_peered":"2023-11-28T11:53:53.355066+0000","last_unstale":"2023-11-28T11:53:54.347631+0000","last_undegraded":"2023-11-28T11:53:54.347631+0000","last_fullsized":"2023-11-28T11:53:54.347631+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:20.156694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347638+0000","last_change":"2023-11-28T11:53:53.357058+0000","last_active":"2023-11-28T11:53:54.347638+0000","last_peered":"2023-11-28T11:53:54.347638+0000","last_clean":"2023-11-28T11:53:54.347638+0000","last_became_active":"2023-11-28T11:53:53.352948+0000","last_became_peered":"2023-11-28T11:53:53.352948+0000","last_unstale":"2023-11-28T11:53:54.347638+0000","last_undegraded":"2023-11-28T11:53:54.347638+0000","last_fullsized":"2023-11-28T11:53:54.347638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:26.522177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347576+0000","last_change":"2023-11-28T11:53:53.356684+0000","last_active":"2023-11-28T11:53:54.347576+0000","last_peered":"2023-11-28T11:53:54.347576+0000","last_clean":"2023-11-28T11:53:54.347576+0000","last_became_active":"2023-11-28T11:53:53.352988+0000","last_became_peered":"2023-11-28T11:53:53.352988+0000","last_unstale":"2023-11-28T11:53:54.347576+0000","last_undegraded":"2023-11-28T11:53:54.347576+0000","last_fullsized":"2023-11-28T11:53:54.347576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:10.953057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365866+0000","last_change":"2023-11-28T11:53:53.362055+0000","last_active":"2023-11-28T11:53:53.365866+0000","last_peered":"2023-11-28T11:53:53.365866+0000","last_clean":"2023-11-28T11:53:53.365866+0000","last_became_active":"2023-11-28T11:53:53.361677+0000","last_became_peered":"2023-11-28T11:53:53.361677+0000","last_unstale":"2023-11-28T11:53:53.365866+0000","last_undegraded":"2023-11-28T11:53:53.365866+0000","last_fullsized":"2023-11-28T11:53:53.365866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:41.839130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348907+0000","last_change":"2023-11-28T11:53:53.362185+0000","last_active":"2023-11-28T11:53:54.348907+0000","last_peered":"2023-11-28T11:53:54.348907+0000","last_clean":"2023-11-28T11:53:54.348907+0000","last_became_active":"2023-11-28T11:53:53.360150+0000","last_became_peered":"2023-11-28T11:53:53.360150+0000","last_unstale":"2023-11-28T11:53:54.348907+0000","last_undegraded":"2023-11-28T11:53:54.348907+0000","last_fullsized":"2023-11-28T11:53:54.348907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:20.242319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347947+0000","last_change":"2023-11-28T11:53:53.354785+0000","last_active":"2023-11-28T11:53:54.347947+0000","last_peered":"2023-11-28T11:53:54.347947+0000","last_clean":"2023-11-28T11:53:54.347947+0000","last_became_active":"2023-11-28T11:53:53.354519+0000","last_became_peered":"2023-11-28T11:53:53.354519+0000","last_unstale":"2023-11-28T11:53:54.347947+0000","last_undegraded":"2023-11-28T11:53:54.347947+0000","last_fullsized":"2023-11-28T11:53:54.347947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:17:18.508153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348557+0000","last_change":"2023-11-28T11:53:53.366035+0000","last_active":"2023-11-28T11:53:54.348557+0000","last_peered":"2023-11-28T11:53:54.348557+0000","last_clean":"2023-11-28T11:53:54.348557+0000","last_became_active":"2023-11-28T11:53:53.365383+0000","last_became_peered":"2023-11-28T11:53:53.365383+0000","last_unstale":"2023-11-28T11:53:54.348557+0000","last_undegraded":"2023-11-28T11:53:54.348557+0000","last_fullsized":"2023-11-28T11:53:54.348557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:42.420815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365520+0000","last_change":"2023-11-28T11:53:53.360229+0000","last_active":"2023-11-28T11:53:53.365520+0000","last_peered":"2023-11-28T11:53:53.365520+0000","last_clean":"2023-11-28T11:53:53.365520+0000","last_became_active":"2023-11-28T11:53:53.359636+0000","last_became_peered":"2023-11-28T11:53:53.359636+0000","last_unstale":"2023-11-28T11:53:53.365520+0000","last_undegraded":"2023-11-28T11:53:53.365520+0000","last_fullsized":"2023-11-28T11:53:53.365520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:44.146115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.371233+0000","last_change":"2023-11-28T11:53:53.367844+0000","last_active":"2023-11-28T11:53:53.371233+0000","last_peered":"2023-11-28T11:53:53.371233+0000","last_clean":"2023-11-28T11:53:53.371233+0000","last_became_active":"2023-11-28T11:53:53.367601+0000","last_became_peered":"2023-11-28T11:53:53.367601+0000","last_unstale":"2023-11-28T11:53:53.371233+0000","last_undegraded":"2023-11-28T11:53:53.371233+0000","last_fullsized":"2023-11-28T11:53:53.371233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:58.872124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347590+0000","last_change":"2023-11-28T11:53:53.357945+0000","last_active":"2023-11-28T11:53:54.347590+0000","last_peered":"2023-11-28T11:53:54.347590+0000","last_clean":"2023-11-28T11:53:54.347590+0000","last_became_active":"2023-11-28T11:53:53.357565+0000","last_became_peered":"2023-11-28T11:53:53.357565+0000","last_unstale":"2023-11-28T11:53:54.347590+0000","last_undegraded":"2023-11-28T11:53:54.347590+0000","last_fullsized":"2023-11-28T11:53:54.347590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:35:00.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348343+0000","last_change":"2023-11-28T11:53:53.365350+0000","last_active":"2023-11-28T11:53:54.348343+0000","last_peered":"2023-11-28T11:53:54.348343+0000","last_clean":"2023-11-28T11:53:54.348343+0000","last_became_active":"2023-11-28T11:53:53.365027+0000","last_became_peered":"2023-11-28T11:53:53.365027+0000","last_unstale":"2023-11-28T11:53:54.348343+0000","last_undegraded":"2023-11-28T11:53:54.348343+0000","last_fullsized":"2023-11-28T11:53:54.348343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:20.156694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823268+0000","last_change":"2023-11-28T11:53:53.357753+0000","last_active":"2023-11-28T11:53:54.823268+0000","last_peered":"2023-11-28T11:53:54.823268+0000","last_clean":"2023-11-28T11:53:54.823268+0000","last_became_active":"2023-11-28T11:53:53.357354+0000","last_became_peered":"2023-11-28T11:53:53.357354+0000","last_unstale":"2023-11-28T11:53:54.823268+0000","last_undegraded":"2023-11-28T11:53:54.823268+0000","last_fullsized":"2023-11-28T11:53:54.823268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:27.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.360469+0000","last_change":"2023-11-28T11:53:53.354154+0000","last_active":"2023-11-28T11:53:53.360469+0000","last_peered":"2023-11-28T11:53:53.360469+0000","last_clean":"2023-11-28T11:53:53.360469+0000","last_became_active":"2023-11-28T11:53:53.353843+0000","last_became_peered":"2023-11-28T11:53:53.353843+0000","last_unstale":"2023-11-28T11:53:53.360469+0000","last_undegraded":"2023-11-28T11:53:53.360469+0000","last_fullsized":"2023-11-28T11:53:53.360469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:57.062194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.369164+0000","last_change":"2023-11-28T11:53:52.940645+0000","last_active":"2023-11-28T11:53:53.369164+0000","last_peered":"2023-11-28T11:53:53.369164+0000","last_clean":"2023-11-28T11:53:53.369164+0000","last_became_active":"2023-11-28T11:53:52.940278+0000","last_became_peered":"2023-11-28T11:53:52.940278+0000","last_unstale":"2023-11-28T11:53:53.369164+0000","last_undegraded":"2023-11-28T11:53:53.369164+0000","last_fullsized":"2023-11-28T11:53:53.369164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:43.982633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348632+0000","last_change":"2023-11-28T11:53:53.365669+0000","last_active":"2023-11-28T11:53:54.348632+0000","last_peered":"2023-11-28T11:53:54.348632+0000","last_clean":"2023-11-28T11:53:54.348632+0000","last_became_active":"2023-11-28T11:53:53.364781+0000","last_became_peered":"2023-11-28T11:53:53.364781+0000","last_unstale":"2023-11-28T11:53:54.348632+0000","last_undegraded":"2023-11-28T11:53:54.348632+0000","last_fullsized":"2023-11-28T11:53:54.348632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:41.839130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823663+0000","last_change":"2023-11-28T11:53:52.948760+0000","last_active":"2023-11-28T11:53:54.823663+0000","last_peered":"2023-11-28T11:53:54.823663+0000","last_clean":"2023-11-28T11:53:54.823663+0000","last_became_active":"2023-11-28T11:53:52.948469+0000","last_became_peered":"2023-11-28T11:53:52.948469+0000","last_unstale":"2023-11-28T11:53:54.823663+0000","last_undegraded":"2023-11-28T11:53:54.823663+0000","last_fullsized":"2023-11-28T11:53:54.823663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:55.310728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823648+0000","last_change":"2023-11-28T11:53:53.362852+0000","last_active":"2023-11-28T11:53:54.823648+0000","last_peered":"2023-11-28T11:53:54.823648+0000","last_clean":"2023-11-28T11:53:54.823648+0000","last_became_active":"2023-11-28T11:53:53.362618+0000","last_became_peered":"2023-11-28T11:53:53.362618+0000","last_unstale":"2023-11-28T11:53:54.823648+0000","last_undegraded":"2023-11-28T11:53:54.823648+0000","last_fullsized":"2023-11-28T11:53:54.823648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:33.067440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823622+0000","last_change":"2023-11-28T11:53:52.947525+0000","last_active":"2023-11-28T11:53:54.823622+0000","last_peered":"2023-11-28T11:53:54.823622+0000","last_clean":"2023-11-28T11:53:54.823622+0000","last_became_active":"2023-11-28T11:53:52.946843+0000","last_became_peered":"2023-11-28T11:53:52.946843+0000","last_unstale":"2023-11-28T11:53:54.823622+0000","last_undegraded":"2023-11-28T11:53:54.823622+0000","last_fullsized":"2023-11-28T11:53:54.823622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:30.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.358477+0000","last_change":"2023-11-28T11:53:53.356387+0000","last_active":"2023-11-28T11:53:53.358477+0000","last_peered":"2023-11-28T11:53:53.358477+0000","last_clean":"2023-11-28T11:53:53.358477+0000","last_became_active":"2023-11-28T11:53:53.355819+0000","last_became_peered":"2023-11-28T11:53:53.355819+0000","last_unstale":"2023-11-28T11:53:53.358477+0000","last_undegraded":"2023-11-28T11:53:53.358477+0000","last_fullsized":"2023-11-28T11:53:53.358477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:28.183193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823542+0000","last_change":"2023-11-28T11:53:52.943916+0000","last_active":"2023-11-28T11:53:54.823542+0000","last_peered":"2023-11-28T11:53:54.823542+0000","last_clean":"2023-11-28T11:53:54.823542+0000","last_became_active":"2023-11-28T11:53:52.943685+0000","last_became_peered":"2023-11-28T11:53:52.943685+0000","last_unstale":"2023-11-28T11:53:54.823542+0000","last_undegraded":"2023-11-28T11:53:54.823542+0000","last_fullsized":"2023-11-28T11:53:54.823542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:17.286301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363641+0000","last_change":"2023-11-28T11:53:53.353957+0000","last_active":"2023-11-28T11:53:53.363641+0000","last_peered":"2023-11-28T11:53:53.363641+0000","last_clean":"2023-11-28T11:53:53.363641+0000","last_became_active":"2023-11-28T11:53:53.353720+0000","last_became_peered":"2023-11-28T11:53:53.353720+0000","last_unstale":"2023-11-28T11:53:53.363641+0000","last_undegraded":"2023-11-28T11:53:53.363641+0000","last_fullsized":"2023-11-28T11:53:53.363641+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:17.297085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347800+0000","last_change":"2023-11-28T11:53:52.944810+0000","last_active":"2023-11-28T11:53:54.347800+0000","last_peered":"2023-11-28T11:53:54.347800+0000","last_clean":"2023-11-28T11:53:54.347800+0000","last_became_active":"2023-11-28T11:53:52.944485+0000","last_became_peered":"2023-11-28T11:53:52.944485+0000","last_unstale":"2023-11-28T11:53:54.347800+0000","last_undegraded":"2023-11-28T11:53:54.347800+0000","last_fullsized":"2023-11-28T11:53:54.347800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:42.354305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367411+0000","last_change":"2023-11-28T11:53:53.365879+0000","last_active":"2023-11-28T11:53:53.367411+0000","last_peered":"2023-11-28T11:53:53.367411+0000","last_clean":"2023-11-28T11:53:53.367411+0000","last_became_active":"2023-11-28T11:53:53.365663+0000","last_became_peered":"2023-11-28T11:53:53.365663+0000","last_unstale":"2023-11-28T11:53:53.367411+0000","last_undegraded":"2023-11-28T11:53:53.367411+0000","last_fullsized":"2023-11-28T11:53:53.367411+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.844830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347624+0000","last_change":"2023-11-28T11:53:52.944839+0000","last_active":"2023-11-28T11:53:54.347624+0000","last_peered":"2023-11-28T11:53:54.347624+0000","last_clean":"2023-11-28T11:53:54.347624+0000","last_became_active":"2023-11-28T11:53:52.944553+0000","last_became_peered":"2023-11-28T11:53:52.944553+0000","last_unstale":"2023-11-28T11:53:54.347624+0000","last_undegraded":"2023-11-28T11:53:54.347624+0000","last_fullsized":"2023-11-28T11:53:54.347624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:14.910583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363430+0000","last_change":"2023-11-28T11:53:53.359147+0000","last_active":"2023-11-28T11:53:53.363430+0000","last_peered":"2023-11-28T11:53:53.363430+0000","last_clean":"2023-11-28T11:53:53.363430+0000","last_became_active":"2023-11-28T11:53:53.358810+0000","last_became_peered":"2023-11-28T11:53:53.358810+0000","last_unstale":"2023-11-28T11:53:53.363430+0000","last_undegraded":"2023-11-28T11:53:53.363430+0000","last_fullsized":"2023-11-28T11:53:53.363430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:32.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347662+0000","last_change":"2023-11-28T11:53:52.942091+0000","last_active":"2023-11-28T11:53:54.347662+0000","last_peered":"2023-11-28T11:53:54.347662+0000","last_clean":"2023-11-28T11:53:54.347662+0000","last_became_active":"2023-11-28T11:53:52.941291+0000","last_became_peered":"2023-11-28T11:53:52.941291+0000","last_unstale":"2023-11-28T11:53:54.347662+0000","last_undegraded":"2023-11-28T11:53:54.347662+0000","last_fullsized":"2023-11-28T11:53:54.347662+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:59.693541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347718+0000","last_change":"2023-11-28T11:53:53.367244+0000","last_active":"2023-11-28T11:53:54.347718+0000","last_peered":"2023-11-28T11:53:54.347718+0000","last_clean":"2023-11-28T11:53:54.347718+0000","last_became_active":"2023-11-28T11:53:53.366967+0000","last_became_peered":"2023-11-28T11:53:53.366967+0000","last_unstale":"2023-11-28T11:53:54.347718+0000","last_undegraded":"2023-11-28T11:53:54.347718+0000","last_fullsized":"2023-11-28T11:53:54.347718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:17.121804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.343023+0000","last_change":"2023-11-28T11:53:52.947769+0000","last_active":"2023-11-28T11:53:53.343023+0000","last_peered":"2023-11-28T11:53:53.343023+0000","last_clean":"2023-11-28T11:53:53.343023+0000","last_became_active":"2023-11-28T11:53:52.947502+0000","last_became_peered":"2023-11-28T11:53:52.947502+0000","last_unstale":"2023-11-28T11:53:53.343023+0000","last_undegraded":"2023-11-28T11:53:53.343023+0000","last_fullsized":"2023-11-28T11:53:53.343023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:29.907062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.361746+0000","last_change":"2023-11-28T11:53:53.353843+0000","last_active":"2023-11-28T11:53:53.361746+0000","last_peered":"2023-11-28T11:53:53.361746+0000","last_clean":"2023-11-28T11:53:53.361746+0000","last_became_active":"2023-11-28T11:53:53.353553+0000","last_became_peered":"2023-11-28T11:53:53.353553+0000","last_unstale":"2023-11-28T11:53:53.361746+0000","last_undegraded":"2023-11-28T11:53:53.361746+0000","last_fullsized":"2023-11-28T11:53:53.361746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:32.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.346649+0000","last_change":"2023-11-28T11:53:52.949620+0000","last_active":"2023-11-28T11:53:53.346649+0000","last_peered":"2023-11-28T11:53:53.346649+0000","last_clean":"2023-11-28T11:53:53.346649+0000","last_became_active":"2023-11-28T11:53:52.949361+0000","last_became_peered":"2023-11-28T11:53:52.949361+0000","last_unstale":"2023-11-28T11:53:53.346649+0000","last_undegraded":"2023-11-28T11:53:53.346649+0000","last_fullsized":"2023-11-28T11:53:53.346649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:59.950858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347793+0000","last_change":"2023-11-28T11:53:53.350460+0000","last_active":"2023-11-28T11:53:54.347793+0000","last_peered":"2023-11-28T11:53:54.347793+0000","last_clean":"2023-11-28T11:53:54.347793+0000","last_became_active":"2023-11-28T11:53:53.350200+0000","last_became_peered":"2023-11-28T11:53:53.350200+0000","last_unstale":"2023-11-28T11:53:54.347793+0000","last_undegraded":"2023-11-28T11:53:54.347793+0000","last_fullsized":"2023-11-28T11:53:54.347793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:01.087022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348736+0000","last_change":"2023-11-28T11:53:52.937666+0000","last_active":"2023-11-28T11:53:54.348736+0000","last_peered":"2023-11-28T11:53:54.348736+0000","last_clean":"2023-11-28T11:53:54.348736+0000","last_became_active":"2023-11-28T11:53:52.937316+0000","last_became_peered":"2023-11-28T11:53:52.937316+0000","last_unstale":"2023-11-28T11:53:54.348736+0000","last_undegraded":"2023-11-28T11:53:54.348736+0000","last_fullsized":"2023-11-28T11:53:54.348736+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:35.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357174+0000","last_change":"2023-11-28T11:53:53.350812+0000","last_active":"2023-11-28T11:53:53.357174+0000","last_peered":"2023-11-28T11:53:53.357174+0000","last_clean":"2023-11-28T11:53:53.357174+0000","last_became_active":"2023-11-28T11:53:53.350172+0000","last_became_peered":"2023-11-28T11:53:53.350172+0000","last_unstale":"2023-11-28T11:53:53.357174+0000","last_undegraded":"2023-11-28T11:53:53.357174+0000","last_fullsized":"2023-11-28T11:53:53.357174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:01.100398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365831+0000","last_change":"2023-11-28T11:53:52.949523+0000","last_active":"2023-11-28T11:53:53.365831+0000","last_peered":"2023-11-28T11:53:53.365831+0000","last_clean":"2023-11-28T11:53:53.365831+0000","last_became_active":"2023-11-28T11:53:52.949294+0000","last_became_peered":"2023-11-28T11:53:52.949294+0000","last_unstale":"2023-11-28T11:53:53.365831+0000","last_undegraded":"2023-11-28T11:53:53.365831+0000","last_fullsized":"2023-11-28T11:53:53.365831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:56.060037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348425+0000","last_change":"2023-11-28T11:53:53.363410+0000","last_active":"2023-11-28T11:53:54.348425+0000","last_peered":"2023-11-28T11:53:54.348425+0000","last_clean":"2023-11-28T11:53:54.348425+0000","last_became_active":"2023-11-28T11:53:53.363127+0000","last_became_peered":"2023-11-28T11:53:53.363127+0000","last_unstale":"2023-11-28T11:53:54.348425+0000","last_undegraded":"2023-11-28T11:53:54.348425+0000","last_fullsized":"2023-11-28T11:53:54.348425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:48.369328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.374665+0000","last_change":"2023-11-28T11:53:52.941884+0000","last_active":"2023-11-28T11:53:53.374665+0000","last_peered":"2023-11-28T11:53:53.374665+0000","last_clean":"2023-11-28T11:53:53.374665+0000","last_became_active":"2023-11-28T11:53:52.941446+0000","last_became_peered":"2023-11-28T11:53:52.941446+0000","last_unstale":"2023-11-28T11:53:53.374665+0000","last_undegraded":"2023-11-28T11:53:53.374665+0000","last_fullsized":"2023-11-28T11:53:53.374665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348036+0000","last_change":"2023-11-28T11:53:53.363425+0000","last_active":"2023-11-28T11:53:54.348036+0000","last_peered":"2023-11-28T11:53:54.348036+0000","last_clean":"2023-11-28T11:53:54.348036+0000","last_became_active":"2023-11-28T11:53:53.363118+0000","last_became_peered":"2023-11-28T11:53:53.363118+0000","last_unstale":"2023-11-28T11:53:54.348036+0000","last_undegraded":"2023-11-28T11:53:54.348036+0000","last_fullsized":"2023-11-28T11:53:54.348036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:58.402661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.364230+0000","last_change":"2023-11-28T11:53:52.947103+0000","last_active":"2023-11-28T11:53:53.364230+0000","last_peered":"2023-11-28T11:53:53.364230+0000","last_clean":"2023-11-28T11:53:53.364230+0000","last_became_active":"2023-11-28T11:53:52.946805+0000","last_became_peered":"2023-11-28T11:53:52.946805+0000","last_unstale":"2023-11-28T11:53:53.364230+0000","last_undegraded":"2023-11-28T11:53:53.364230+0000","last_fullsized":"2023-11-28T11:53:53.364230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:37.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347328+0000","last_change":"2023-11-28T11:53:53.357625+0000","last_active":"2023-11-28T11:53:54.347328+0000","last_peered":"2023-11-28T11:53:54.347328+0000","last_clean":"2023-11-28T11:53:54.347328+0000","last_became_active":"2023-11-28T11:53:53.357330+0000","last_became_peered":"2023-11-28T11:53:53.357330+0000","last_unstale":"2023-11-28T11:53:54.347328+0000","last_undegraded":"2023-11-28T11:53:54.347328+0000","last_fullsized":"2023-11-28T11:53:54.347328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:37.461966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347683+0000","last_change":"2023-11-28T11:53:52.946327+0000","last_active":"2023-11-28T11:53:54.347683+0000","last_peered":"2023-11-28T11:53:54.347683+0000","last_clean":"2023-11-28T11:53:54.347683+0000","last_became_active":"2023-11-28T11:53:52.946031+0000","last_became_peered":"2023-11-28T11:53:52.946031+0000","last_unstale":"2023-11-28T11:53:54.347683+0000","last_undegraded":"2023-11-28T11:53:54.347683+0000","last_fullsized":"2023-11-28T11:53:54.347683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:31.298250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357675+0000","last_change":"2023-11-28T11:53:53.352629+0000","last_active":"2023-11-28T11:53:53.357675+0000","last_peered":"2023-11-28T11:53:53.357675+0000","last_clean":"2023-11-28T11:53:53.357675+0000","last_became_active":"2023-11-28T11:53:53.352392+0000","last_became_peered":"2023-11-28T11:53:53.352392+0000","last_unstale":"2023-11-28T11:53:53.357675+0000","last_undegraded":"2023-11-28T11:53:53.357675+0000","last_fullsized":"2023-11-28T11:53:53.357675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:44.146115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.346091+0000","last_change":"2023-11-28T11:53:52.945727+0000","last_active":"2023-11-28T11:53:53.346091+0000","last_peered":"2023-11-28T11:53:53.346091+0000","last_clean":"2023-11-28T11:53:53.346091+0000","last_became_active":"2023-11-28T11:53:52.945394+0000","last_became_peered":"2023-11-28T11:53:52.945394+0000","last_unstale":"2023-11-28T11:53:53.346091+0000","last_undegraded":"2023-11-28T11:53:53.346091+0000","last_fullsized":"2023-11-28T11:53:53.346091+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:20.339134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.364921+0000","last_change":"2023-11-28T11:53:53.360149+0000","last_active":"2023-11-28T11:53:53.364921+0000","last_peered":"2023-11-28T11:53:53.364921+0000","last_clean":"2023-11-28T11:53:53.364921+0000","last_became_active":"2023-11-28T11:53:53.359902+0000","last_became_peered":"2023-11-28T11:53:53.359902+0000","last_unstale":"2023-11-28T11:53:53.364921+0000","last_undegraded":"2023-11-28T11:53:53.364921+0000","last_fullsized":"2023-11-28T11:53:53.364921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:48.369328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348058+0000","last_change":"2023-11-28T11:53:52.940538+0000","last_active":"2023-11-28T11:53:54.348058+0000","last_peered":"2023-11-28T11:53:54.348058+0000","last_clean":"2023-11-28T11:53:54.348058+0000","last_became_active":"2023-11-28T11:53:52.940228+0000","last_became_peered":"2023-11-28T11:53:52.940228+0000","last_unstale":"2023-11-28T11:53:54.348058+0000","last_undegraded":"2023-11-28T11:53:54.348058+0000","last_fullsized":"2023-11-28T11:53:54.348058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:10.424141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823547+0000","last_change":"2023-11-28T11:53:53.367540+0000","last_active":"2023-11-28T11:53:54.823547+0000","last_peered":"2023-11-28T11:53:54.823547+0000","last_clean":"2023-11-28T11:53:54.823547+0000","last_became_active":"2023-11-28T11:53:53.367276+0000","last_became_peered":"2023-11-28T11:53:53.367276+0000","last_unstale":"2023-11-28T11:53:54.823547+0000","last_undegraded":"2023-11-28T11:53:54.823547+0000","last_fullsized":"2023-11-28T11:53:54.823547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:25:42.073775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348942+0000","last_change":"2023-11-28T11:53:52.947823+0000","last_active":"2023-11-28T11:53:54.348942+0000","last_peered":"2023-11-28T11:53:54.348942+0000","last_clean":"2023-11-28T11:53:54.348942+0000","last_became_active":"2023-11-28T11:53:52.947513+0000","last_became_peered":"2023-11-28T11:53:52.947513+0000","last_unstale":"2023-11-28T11:53:54.348942+0000","last_undegraded":"2023-11-28T11:53:54.348942+0000","last_fullsized":"2023-11-28T11:53:54.348942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:20.339134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.362443+0000","last_change":"2023-11-28T11:53:53.357944+0000","last_active":"2023-11-28T11:53:53.362443+0000","last_peered":"2023-11-28T11:53:53.362443+0000","last_clean":"2023-11-28T11:53:53.362443+0000","last_became_active":"2023-11-28T11:53:53.357630+0000","last_became_peered":"2023-11-28T11:53:53.357630+0000","last_unstale":"2023-11-28T11:53:53.362443+0000","last_undegraded":"2023-11-28T11:53:53.362443+0000","last_fullsized":"2023-11-28T11:53:53.362443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:22:59.644754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348118+0000","last_change":"2023-11-28T11:53:52.947074+0000","last_active":"2023-11-28T11:53:54.348118+0000","last_peered":"2023-11-28T11:53:54.348118+0000","last_clean":"2023-11-28T11:53:54.348118+0000","last_became_active":"2023-11-28T11:53:52.946804+0000","last_became_peered":"2023-11-28T11:53:52.946804+0000","last_unstale":"2023-11-28T11:53:54.348118+0000","last_undegraded":"2023-11-28T11:53:54.348118+0000","last_fullsized":"2023-11-28T11:53:54.348118+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:59.950858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348040+0000","last_change":"2023-11-28T11:53:53.362038+0000","last_active":"2023-11-28T11:53:54.348040+0000","last_peered":"2023-11-28T11:53:54.348040+0000","last_clean":"2023-11-28T11:53:54.348040+0000","last_became_active":"2023-11-28T11:53:53.361428+0000","last_became_peered":"2023-11-28T11:53:53.361428+0000","last_unstale":"2023-11-28T11:53:54.348040+0000","last_undegraded":"2023-11-28T11:53:54.348040+0000","last_fullsized":"2023-11-28T11:53:54.348040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:30.442863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823652+0000","last_change":"2023-11-28T11:53:52.945655+0000","last_active":"2023-11-28T11:53:54.823652+0000","last_peered":"2023-11-28T11:53:54.823652+0000","last_clean":"2023-11-28T11:53:54.823652+0000","last_became_active":"2023-11-28T11:53:52.945344+0000","last_became_peered":"2023-11-28T11:53:52.945344+0000","last_unstale":"2023-11-28T11:53:54.823652+0000","last_undegraded":"2023-11-28T11:53:54.823652+0000","last_fullsized":"2023-11-28T11:53:54.823652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:45.264482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357059+0000","last_change":"2023-11-28T11:53:53.351406+0000","last_active":"2023-11-28T11:53:53.357059+0000","last_peered":"2023-11-28T11:53:53.357059+0000","last_clean":"2023-11-28T11:53:53.357059+0000","last_became_active":"2023-11-28T11:53:53.350803+0000","last_became_peered":"2023-11-28T11:53:53.350803+0000","last_unstale":"2023-11-28T11:53:53.357059+0000","last_undegraded":"2023-11-28T11:53:53.357059+0000","last_fullsized":"2023-11-28T11:53:53.357059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:21.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.354302+0000","last_change":"2023-11-28T11:53:52.941710+0000","last_active":"2023-11-28T11:53:53.354302+0000","last_peered":"2023-11-28T11:53:53.354302+0000","last_clean":"2023-11-28T11:53:53.354302+0000","last_became_active":"2023-11-28T11:53:52.941111+0000","last_became_peered":"2023-11-28T11:53:52.941111+0000","last_unstale":"2023-11-28T11:53:53.354302+0000","last_undegraded":"2023-11-28T11:53:53.354302+0000","last_fullsized":"2023-11-28T11:53:53.354302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:42.644755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.360999+0000","last_change":"2023-11-28T11:53:53.356586+0000","last_active":"2023-11-28T11:53:53.360999+0000","last_peered":"2023-11-28T11:53:53.360999+0000","last_clean":"2023-11-28T11:53:53.360999+0000","last_became_active":"2023-11-28T11:53:53.356292+0000","last_became_peered":"2023-11-28T11:53:53.356292+0000","last_unstale":"2023-11-28T11:53:53.360999+0000","last_undegraded":"2023-11-28T11:53:53.360999+0000","last_fullsized":"2023-11-28T11:53:53.360999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:07.501442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.346351+0000","last_change":"2023-11-28T11:53:52.945663+0000","last_active":"2023-11-28T11:53:53.346351+0000","last_peered":"2023-11-28T11:53:53.346351+0000","last_clean":"2023-11-28T11:53:53.346351+0000","last_became_active":"2023-11-28T11:53:52.945389+0000","last_became_peered":"2023-11-28T11:53:52.945389+0000","last_unstale":"2023-11-28T11:53:53.346351+0000","last_undegraded":"2023-11-28T11:53:53.346351+0000","last_fullsized":"2023-11-28T11:53:53.346351+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.499284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348469+0000","last_change":"2023-11-28T11:53:53.363683+0000","last_active":"2023-11-28T11:53:54.348469+0000","last_peered":"2023-11-28T11:53:54.348469+0000","last_clean":"2023-11-28T11:53:54.348469+0000","last_became_active":"2023-11-28T11:53:53.363336+0000","last_became_peered":"2023-11-28T11:53:53.363336+0000","last_unstale":"2023-11-28T11:53:54.348469+0000","last_undegraded":"2023-11-28T11:53:54.348469+0000","last_fullsized":"2023-11-28T11:53:54.348469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:24:40.402636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347720+0000","last_change":"2023-11-28T11:53:52.937233+0000","last_active":"2023-11-28T11:53:54.347720+0000","last_peered":"2023-11-28T11:53:54.347720+0000","last_clean":"2023-11-28T11:53:54.347720+0000","last_became_active":"2023-11-28T11:53:52.936952+0000","last_became_peered":"2023-11-28T11:53:52.936952+0000","last_unstale":"2023-11-28T11:53:54.347720+0000","last_undegraded":"2023-11-28T11:53:54.347720+0000","last_fullsized":"2023-11-28T11:53:54.347720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:46.490834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367320+0000","last_change":"2023-11-28T11:53:53.365718+0000","last_active":"2023-11-28T11:53:53.367320+0000","last_peered":"2023-11-28T11:53:53.367320+0000","last_clean":"2023-11-28T11:53:53.367320+0000","last_became_active":"2023-11-28T11:53:53.365501+0000","last_became_peered":"2023-11-28T11:53:53.365501+0000","last_unstale":"2023-11-28T11:53:53.367320+0000","last_undegraded":"2023-11-28T11:53:53.367320+0000","last_fullsized":"2023-11-28T11:53:53.367320+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:01.100398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347644+0000","last_change":"2023-11-28T11:53:52.940103+0000","last_active":"2023-11-28T11:53:54.347644+0000","last_peered":"2023-11-28T11:53:54.347644+0000","last_clean":"2023-11-28T11:53:54.347644+0000","last_became_active":"2023-11-28T11:53:52.939842+0000","last_became_peered":"2023-11-28T11:53:52.939842+0000","last_unstale":"2023-11-28T11:53:54.347644+0000","last_undegraded":"2023-11-28T11:53:54.347644+0000","last_fullsized":"2023-11-28T11:53:54.347644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:41.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.361370+0000","last_change":"2023-11-28T11:53:53.353001+0000","last_active":"2023-11-28T11:53:53.361370+0000","last_peered":"2023-11-28T11:53:53.361370+0000","last_clean":"2023-11-28T11:53:53.361370+0000","last_became_active":"2023-11-28T11:53:53.352771+0000","last_became_peered":"2023-11-28T11:53:53.352771+0000","last_unstale":"2023-11-28T11:53:53.361370+0000","last_undegraded":"2023-11-28T11:53:53.361370+0000","last_fullsized":"2023-11-28T11:53:53.361370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:55.715676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.353965+0000","last_change":"2023-11-28T11:53:52.943760+0000","last_active":"2023-11-28T11:53:53.353965+0000","last_peered":"2023-11-28T11:53:53.353965+0000","last_clean":"2023-11-28T11:53:53.353965+0000","last_became_active":"2023-11-28T11:53:52.943484+0000","last_became_peered":"2023-11-28T11:53:52.943484+0000","last_unstale":"2023-11-28T11:53:53.353965+0000","last_undegraded":"2023-11-28T11:53:53.353965+0000","last_fullsized":"2023-11-28T11:53:53.353965+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:39.945555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348626+0000","last_change":"2023-11-28T11:53:53.363345+0000","last_active":"2023-11-28T11:53:54.348626+0000","last_peered":"2023-11-28T11:53:54.348626+0000","last_clean":"2023-11-28T11:53:54.348626+0000","last_became_active":"2023-11-28T11:53:53.363053+0000","last_became_peered":"2023-11-28T11:53:53.363053+0000","last_unstale":"2023-11-28T11:53:54.348626+0000","last_undegraded":"2023-11-28T11:53:54.348626+0000","last_fullsized":"2023-11-28T11:53:54.348626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:56.344374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365792+0000","last_change":"2023-11-28T11:53:52.949669+0000","last_active":"2023-11-28T11:53:53.365792+0000","last_peered":"2023-11-28T11:53:53.365792+0000","last_clean":"2023-11-28T11:53:53.365792+0000","last_became_active":"2023-11-28T11:53:52.949404+0000","last_became_peered":"2023-11-28T11:53:52.949404+0000","last_unstale":"2023-11-28T11:53:53.365792+0000","last_undegraded":"2023-11-28T11:53:53.365792+0000","last_fullsized":"2023-11-28T11:53:53.365792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:35.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.355760+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:53:53.355760+0000","last_peered":"2023-11-28T11:53:53.355760+0000","last_clean":"2023-11-28T11:53:53.355760+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:53:53.355760+0000","last_undegraded":"2023-11-28T11:53:53.355760+0000","last_fullsized":"2023-11-28T11:53:53.355760+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357084+0000","last_change":"2023-11-28T11:53:53.351618+0000","last_active":"2023-11-28T11:53:53.357084+0000","last_peered":"2023-11-28T11:53:53.357084+0000","last_clean":"2023-11-28T11:53:53.357084+0000","last_became_active":"2023-11-28T11:53:53.351148+0000","last_became_peered":"2023-11-28T11:53:53.351148+0000","last_unstale":"2023-11-28T11:53:53.357084+0000","last_undegraded":"2023-11-28T11:53:53.357084+0000","last_fullsized":"2023-11-28T11:53:53.357084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:29:00.953015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347779+0000","last_change":"2023-11-28T11:53:52.939287+0000","last_active":"2023-11-28T11:53:54.347779+0000","last_peered":"2023-11-28T11:53:54.347779+0000","last_clean":"2023-11-28T11:53:54.347779+0000","last_became_active":"2023-11-28T11:53:52.938993+0000","last_became_peered":"2023-11-28T11:53:52.938993+0000","last_unstale":"2023-11-28T11:53:54.347779+0000","last_undegraded":"2023-11-28T11:53:54.347779+0000","last_fullsized":"2023-11-28T11:53:54.347779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:27:42.936931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.345063+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:53:53.345063+0000","last_peered":"2023-11-28T11:53:53.345063+0000","last_clean":"2023-11-28T11:53:53.345063+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:53:53.345063+0000","last_undegraded":"2023-11-28T11:53:53.345063+0000","last_fullsized":"2023-11-28T11:53:53.345063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823334+0000","last_change":"2023-11-28T11:53:53.357600+0000","last_active":"2023-11-28T11:53:54.823334+0000","last_peered":"2023-11-28T11:53:54.823334+0000","last_clean":"2023-11-28T11:53:54.823334+0000","last_became_active":"2023-11-28T11:53:53.357045+0000","last_became_peered":"2023-11-28T11:53:53.357045+0000","last_unstale":"2023-11-28T11:53:54.823334+0000","last_undegraded":"2023-11-28T11:53:54.823334+0000","last_fullsized":"2023-11-28T11:53:54.823334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.873940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823410+0000","last_change":"2023-11-28T11:53:52.946969+0000","last_active":"2023-11-28T11:53:54.823410+0000","last_peered":"2023-11-28T11:53:54.823410+0000","last_clean":"2023-11-28T11:53:54.823410+0000","last_became_active":"2023-11-28T11:53:52.946640+0000","last_became_peered":"2023-11-28T11:53:52.946640+0000","last_unstale":"2023-11-28T11:53:54.823410+0000","last_undegraded":"2023-11-28T11:53:54.823410+0000","last_fullsized":"2023-11-28T11:53:54.823410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:00.844589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":42,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.930508+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:53:52.930508+0000","last_peered":"2023-11-28T11:53:52.930508+0000","last_clean":"2023-11-28T11:53:52.930508+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:53:52.930508+0000","last_undegraded":"2023-11-28T11:53:52.930508+0000","last_fullsized":"2023-11-28T11:53:52.930508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":193,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.936228+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:53:52.936228+0000","last_peered":"2023-11-28T11:53:52.936228+0000","last_clean":"2023-11-28T11:53:52.936228+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:53:52.936228+0000","last_undegraded":"2023-11-28T11:53:52.936228+0000","last_fullsized":"2023-11-28T11:53:52.936228+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347470+0000","last_change":"2023-11-28T11:53:53.354642+0000","last_active":"2023-11-28T11:53:54.347470+0000","last_peered":"2023-11-28T11:53:54.347470+0000","last_clean":"2023-11-28T11:53:54.347470+0000","last_became_active":"2023-11-28T11:53:53.354347+0000","last_became_peered":"2023-11-28T11:53:53.354347+0000","last_unstale":"2023-11-28T11:53:54.347470+0000","last_undegraded":"2023-11-28T11:53:54.347470+0000","last_fullsized":"2023-11-28T11:53:54.347470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:42.420815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347763+0000","last_change":"2023-11-28T11:53:52.944877+0000","last_active":"2023-11-28T11:53:54.347763+0000","last_peered":"2023-11-28T11:53:54.347763+0000","last_clean":"2023-11-28T11:53:54.347763+0000","last_became_active":"2023-11-28T11:53:52.944176+0000","last_became_peered":"2023-11-28T11:53:52.944176+0000","last_unstale":"2023-11-28T11:53:54.347763+0000","last_undegraded":"2023-11-28T11:53:54.347763+0000","last_fullsized":"2023-11-28T11:53:54.347763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:00.098240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.938643+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:53:52.938643+0000","last_peered":"2023-11-28T11:53:52.938643+0000","last_clean":"2023-11-28T11:53:52.938643+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:53:52.938643+0000","last_undegraded":"2023-11-28T11:53:52.938643+0000","last_fullsized":"2023-11-28T11:53:52.938643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.361032+0000","last_change":"2023-11-28T11:53:53.354352+0000","last_active":"2023-11-28T11:53:53.361032+0000","last_peered":"2023-11-28T11:53:53.361032+0000","last_clean":"2023-11-28T11:53:53.361032+0000","last_became_active":"2023-11-28T11:53:53.354105+0000","last_became_peered":"2023-11-28T11:53:53.354105+0000","last_unstale":"2023-11-28T11:53:53.361032+0000","last_undegraded":"2023-11-28T11:53:53.361032+0000","last_fullsized":"2023-11-28T11:53:53.361032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:36.169075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347564+0000","last_change":"2023-11-28T11:53:52.944777+0000","last_active":"2023-11-28T11:53:54.347564+0000","last_peered":"2023-11-28T11:53:54.347564+0000","last_clean":"2023-11-28T11:53:54.347564+0000","last_became_active":"2023-11-28T11:53:52.944431+0000","last_became_peered":"2023-11-28T11:53:52.944431+0000","last_unstale":"2023-11-28T11:53:54.347564+0000","last_undegraded":"2023-11-28T11:53:54.347564+0000","last_fullsized":"2023-11-28T11:53:54.347564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:00.334786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.936124+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:53:52.936124+0000","last_peered":"2023-11-28T11:53:52.936124+0000","last_clean":"2023-11-28T11:53:52.936124+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:53:52.936124+0000","last_undegraded":"2023-11-28T11:53:52.936124+0000","last_fullsized":"2023-11-28T11:53:52.936124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363412+0000","last_change":"2023-11-28T11:53:53.357550+0000","last_active":"2023-11-28T11:53:53.363412+0000","last_peered":"2023-11-28T11:53:53.363412+0000","last_clean":"2023-11-28T11:53:53.363412+0000","last_became_active":"2023-11-28T11:53:53.356972+0000","last_became_peered":"2023-11-28T11:53:53.356972+0000","last_unstale":"2023-11-28T11:53:53.363412+0000","last_undegraded":"2023-11-28T11:53:53.363412+0000","last_fullsized":"2023-11-28T11:53:53.363412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:06.472324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823352+0000","last_change":"2023-11-28T11:53:52.948014+0000","last_active":"2023-11-28T11:53:54.823352+0000","last_peered":"2023-11-28T11:53:54.823352+0000","last_clean":"2023-11-28T11:53:54.823352+0000","last_became_active":"2023-11-28T11:53:52.947761+0000","last_became_peered":"2023-11-28T11:53:52.947761+0000","last_unstale":"2023-11-28T11:53:54.823352+0000","last_undegraded":"2023-11-28T11:53:54.823352+0000","last_fullsized":"2023-11-28T11:53:54.823352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:30:59.358348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.348383+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:53:53.348383+0000","last_peered":"2023-11-28T11:53:53.348383+0000","last_clean":"2023-11-28T11:53:53.348383+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:53:53.348383+0000","last_undegraded":"2023-11-28T11:53:53.348383+0000","last_fullsized":"2023-11-28T11:53:53.348383+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.371693+0000","last_change":"2023-11-28T11:53:53.368465+0000","last_active":"2023-11-28T11:53:53.371693+0000","last_peered":"2023-11-28T11:53:53.371693+0000","last_clean":"2023-11-28T11:53:53.371693+0000","last_became_active":"2023-11-28T11:53:53.367541+0000","last_became_peered":"2023-11-28T11:53:53.367541+0000","last_unstale":"2023-11-28T11:53:53.371693+0000","last_undegraded":"2023-11-28T11:53:53.371693+0000","last_fullsized":"2023-11-28T11:53:53.371693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.844830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367522+0000","last_change":"2023-11-28T11:53:52.945782+0000","last_active":"2023-11-28T11:53:53.367522+0000","last_peered":"2023-11-28T11:53:53.367522+0000","last_clean":"2023-11-28T11:53:53.367522+0000","last_became_active":"2023-11-28T11:53:52.945501+0000","last_became_peered":"2023-11-28T11:53:52.945501+0000","last_unstale":"2023-11-28T11:53:53.367522+0000","last_undegraded":"2023-11-28T11:53:53.367522+0000","last_fullsized":"2023-11-28T11:53:53.367522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:10.424141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.937142+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:53:52.937142+0000","last_peered":"2023-11-28T11:53:52.937142+0000","last_clean":"2023-11-28T11:53:52.937142+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:53:52.937142+0000","last_undegraded":"2023-11-28T11:53:52.937142+0000","last_fullsized":"2023-11-28T11:53:52.937142+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366542+0000","last_change":"2023-11-28T11:53:53.365132+0000","last_active":"2023-11-28T11:53:53.366542+0000","last_peered":"2023-11-28T11:53:53.366542+0000","last_clean":"2023-11-28T11:53:53.366542+0000","last_became_active":"2023-11-28T11:53:53.364841+0000","last_became_peered":"2023-11-28T11:53:53.364841+0000","last_unstale":"2023-11-28T11:53:53.366542+0000","last_undegraded":"2023-11-28T11:53:53.366542+0000","last_fullsized":"2023-11-28T11:53:53.366542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:18.360820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348816+0000","last_change":"2023-11-28T11:53:52.944615+0000","last_active":"2023-11-28T11:53:54.348816+0000","last_peered":"2023-11-28T11:53:54.348816+0000","last_clean":"2023-11-28T11:53:54.348816+0000","last_became_active":"2023-11-28T11:53:52.944325+0000","last_became_peered":"2023-11-28T11:53:52.944325+0000","last_unstale":"2023-11-28T11:53:54.348816+0000","last_undegraded":"2023-11-28T11:53:54.348816+0000","last_fullsized":"2023-11-28T11:53:54.348816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:05.470167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.937086+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:53:52.937086+0000","last_peered":"2023-11-28T11:53:52.937086+0000","last_clean":"2023-11-28T11:53:52.937086+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:53:52.937086+0000","last_undegraded":"2023-11-28T11:53:52.937086+0000","last_fullsized":"2023-11-28T11:53:52.937086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10700,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361140,"statfs":{"total":96636764160,"available":96625807360,"internally_reserved":0,"allocated":1314816,"data_stored":898513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.45700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.62},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.66600000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.46300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70199999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.497},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74099999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61099999999999999,"5min":0.751,"15min":0.751},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.502,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72599999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71799999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":708608,"data_stored":307419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.61399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.629},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70999999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":3.7810000000000001,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73699999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.46300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.44500000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51800000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10456,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361384,"statfs":{"total":96636764160,"available":96626057216,"internally_reserved":0,"allocated":1327104,"data_stored":900049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.42399999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35299999999999998,"15min":0.35199999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.499,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.30499999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38300000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.44,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80200000000000005,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48299999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13568,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358272,"statfs":{"total":96636764160,"available":96622870528,"internally_reserved":0,"allocated":712704,"data_stored":313052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.316,"15min":0.31},"min":{"1min":0.22,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.436},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.34899999999999998},"min":{"1min":0.253,"5min":0.251,"15min":0.251},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.29299999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.30099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.44600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.49099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.505}]},{"osd":7,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.502,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51700000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9336,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362504,"statfs":{"total":96636764160,"available":96627204096,"internally_reserved":0,"allocated":704512,"data_stored":307385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47999999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.66800000000000004,"15min":0.71599999999999997},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.57299999999999995,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.434}]},{"osd":1,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.30599999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48999999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.68700000000000006,"15min":0.73599999999999999},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34200000000000003,"15min":0.33600000000000002},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.67700000000000005,"15min":0.71899999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57599999999999996,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.91700000000000004,"15min":0.96399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":8.359,"15min":8.359},"last":0.749}]},{"osd":5,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.86199999999999999,"15min":0.89800000000000002},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82999999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.90400000000000003,"15min":0.94599999999999995},"min":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84899999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.75800000000000001,"15min":0.78000000000000003},"min":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.79100000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9720,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362120,"statfs":{"total":96636764160,"available":96626810880,"internally_reserved":0,"allocated":704512,"data_stored":307385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.25,"5min":0.224,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":1.762,"15min":1.762},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.317}]},{"osd":1,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.96699999999999997,"15min":1.0669999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55500000000000005,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.29199999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61899999999999999,"5min":1.857,"15min":1.857},"last":0.33700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.56200000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.70299999999999996,"15min":0.749},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60699999999999998,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88700000000000001,"5min":1.498,"15min":1.498},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.66300000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86699999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98999999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.55000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9540,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362300,"statfs":{"total":96636764160,"available":96626995200,"internally_reserved":0,"allocated":716800,"data_stored":308933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67300000000000004,"5min":2.137,"15min":2.137},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.5},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.378,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59399999999999997,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69699999999999995,"5min":0.745,"15min":0.745},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.629,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.35799999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.30499999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65500000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.433,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.497}]},{"osd":6,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.60699999999999998,"15min":0.623},"min":{"1min":0.36499999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71799999999999997,"5min":2.286,"15min":2.286},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70799999999999996,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.64800000000000002,"15min":0.66400000000000003},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67300000000000004,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53700000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9400,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362440,"statfs":{"total":96636764160,"available":96627138560,"internally_reserved":0,"allocated":704512,"data_stored":306421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77400000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.40100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79200000000000004,"5min":1.679,"15min":1.679},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.504}]},{"osd":4,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.373,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79400000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.39000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.623},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.371,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76000000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:53:57.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:53:57.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.577+0000 7f5e5b5e7700 1 Processor -- start 2023-11-28T11:53:57.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.577+0000 7f5e5b5e7700 1 -- start start 2023-11-28T11:53:57.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e5b5e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e54071040 0x7f5e54071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e5b5e7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e54071950 con 0x7f5e5406dee0 2023-11-28T11:53:57.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e5b5e7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5415d060 con 0x7f5e54160540 2023-11-28T11:53:57.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e5b5e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5415d340 con 0x7f5e54071040 2023-11-28T11:53:57.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e58b82700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5e5406dee0 0x7f5e5406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55194/0 (socket says 172.21.15.144:55194) 2023-11-28T11:53:57.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e58b82700 1 -- 172.21.15.144:0/2274724432 learned_addr learned my addr 172.21.15.144:0/2274724432 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:57.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422666919 0 0) 0x7f5e54071950 con 0x7f5e5406dee0 2023-11-28T11:53:57.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e400035b0 con 0x7f5e5406dee0 2023-11-28T11:53:57.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e59383700 1 --2- 172.21.15.144:0/2274724432 >> v2:172.21.15.177:3300/0 conn(0x7f5e54071040 0x7f5e54071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.578+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063198000 0 0) 0x7f5e400035b0 con 0x7f5e5406dee0 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 >> v2:172.21.15.177:3300/0 conn(0x7f5e54071040 msgr2=0x7f5e54071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 --2- 172.21.15.144:0/2274724432 >> v2:172.21.15.177:3300/0 conn(0x7f5e54071040 0x7f5e54071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 >> v1:172.21.15.177:6789/0 conn(0x7f5e54160540 legacy=0x7f5e54160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5415d620 con 0x7f5e5406dee0 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f5e50001670 con 0x7f5e5406dee0 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59383700 1 --2- 172.21.15.144:0/2274724432 >> v2:172.21.15.177:3300/0 conn(0x7f5e54071040 0x7f5e54071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 >> v1:172.21.15.144:6789/0 conn(0x7f5e5406dee0 legacy=0x7f5e5406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e40003a00 0x7f5e40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40004850 0x7f5e40004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e40005400 0x7f5e4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5415d340 con 0x7f5e40005400 2023-11-28T11:53:57.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4000c280 con 0x7f5e40004850 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4000c440 con 0x7f5e40003a00 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e40005400 0x7f5e4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e58b82700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40004850 0x7f5e40004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59383700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e40003a00 0x7f5e40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 -- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e40003a00 msgr2=0x7f5e40003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e40003a00 0x7f5e40003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 -- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40004850 msgr2=0x7f5e40004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40004850 0x7f5e40004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 -- 172.21.15.144:0/2274724432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400035b0 con 0x7f5e40005400 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 -- 172.21.15.144:0/2274724432 reap_dead start 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e59b84700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e40005400 0x7f5e4000ba10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c240 tx=0x7f5e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b2bcbae441aed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c001df0 con 0x7f5e40005400 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c00cda0 con 0x7f5e40005400 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.579+0000 7f5e4bfff700 1 -- 172.21.15.144:0/2274724432 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c003c40 con 0x7f5e40005400 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/2274724432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e40005400 msgr2=0x7f5e4000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e40005400 0x7f5e4000ba10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c240 tx=0x7f5e40005880 comp rx=0 tx=0).stop 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/2274724432 shutdown_connections 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/2274724432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e40005400 0x7f5e4000ba10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/2274724432 >> 172.21.15.144:0/2274724432 conn(0x7f5e540d0000 msgr2=0x7f5e540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/2274724432 shutdown_connections 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.580+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/2274724432 wait complete. 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.581+0000 7f5e5b5e7700 1 Processor -- start 2023-11-28T11:53:57.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 -- start start 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40005400 0x7f5e541f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 0x7f5e540b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e540b5a60 0x7f5e540c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e540b5ed0 con 0x7f5e54157200 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e541561f0 con 0x7f5e40005400 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.582+0000 7f5e5b5e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e54156440 con 0x7f5e540b5a60 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 0x7f5e540b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 0x7f5e540b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38494/0 (socket says 172.21.15.144:38494) 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 -- 172.21.15.144:0/3892619106 learned_addr learned my addr 172.21.15.144:0/3892619106 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:57.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e59b84700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e540b5a60 0x7f5e540c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e59383700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40005400 0x7f5e541f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 -- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e540b5a60 msgr2=0x7f5e540c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e540b5a60 0x7f5e540c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 -- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40005400 msgr2=0x7f5e541f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40005400 0x7f5e541f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 -- 172.21.15.144:0/3892619106 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5415ccd0 con 0x7f5e54157200 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e58b82700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 0x7f5e540b5520 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5e500072f0 tx=0x7f5e500076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500077d0 con 0x7f5e54157200 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5000f870 con 0x7f5e54157200 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5415d900 con 0x7f5e54157200 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000fc20 con 0x7f5e54157200 2023-11-28T11:53:57.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.583+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5406a270 con 0x7f5e54157200 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.584+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e540720a0 con 0x7f5e54157200 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.585+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e5001c030 con 0x7f5e54157200 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.585+0000 7f5e49ffb700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e380682f0 0x7f5e3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.585+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e500854d0 con 0x7f5e54157200 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.585+0000 7f5e59383700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e380682f0 0x7f5e3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.585+0000 7f5e59383700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e380682f0 0x7f5e3806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5e44004470 tx=0x7f5e44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.589+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5e50051cd0 con 0x7f5e54157200 2023-11-28T11:53:57.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.766+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e5406f440 con 0x7f5e380682f0 2023-11-28T11:53:57.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.775+0000 7f5e49ffb700 1 -- 172.21.15.144:0/3892619106 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311897 (secure 0 0 0) 0x7f5e5406f440 con 0x7f5e380682f0 2023-11-28T11:53:57.780 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e380682f0 msgr2=0x7f5e3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e380682f0 0x7f5e3806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5e44004470 tx=0x7f5e44008040 comp rx=0 tx=0).stop 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 msgr2=0x7f5e540b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 0x7f5e540b5520 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5e500072f0 tx=0x7f5e500076b0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 shutdown_connections 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e380682f0 0x7f5e3806a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.783 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e540b5a60 0x7f5e540c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e40005400 0x7f5e541f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 --2- 172.21.15.144:0/3892619106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e54157200 0x7f5e540b5520 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 >> 172.21.15.144:0/3892619106 conn(0x7f5e540d0000 msgr2=0x7f5e5415fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 shutdown_connections 2023-11-28T11:53:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.778+0000 7f5e5b5e7700 1 -- 172.21.15.144:0/3892619106 wait complete. 2023-11-28T11:53:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:53:57.792 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2023-11-28T11:53:56.517880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81996,"kb_used_data":6732,"kb_used_omap":0,"kb_used_meta":75200,"kb_avail":754892724,"statfs":{"total":773094113280,"available":773010149376,"internally_reserved":0,"allocated":6893568,"data_stored":3649157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":77004800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.591716"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365099+0000","last_change":"2023-11-28T11:53:53.355794+0000","last_active":"2023-11-28T11:53:53.365099+0000","last_peered":"2023-11-28T11:53:53.365099+0000","last_clean":"2023-11-28T11:53:53.365099+0000","last_became_active":"2023-11-28T11:53:53.355336+0000","last_became_peered":"2023-11-28T11:53:53.355336+0000","last_unstale":"2023-11-28T11:53:53.365099+0000","last_undegraded":"2023-11-28T11:53:53.365099+0000","last_fullsized":"2023-11-28T11:53:53.365099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:19.117644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367391+0000","last_change":"2023-11-28T11:53:53.364706+0000","last_active":"2023-11-28T11:53:53.367391+0000","last_peered":"2023-11-28T11:53:53.367391+0000","last_clean":"2023-11-28T11:53:53.367391+0000","last_became_active":"2023-11-28T11:53:53.364111+0000","last_became_peered":"2023-11-28T11:53:53.364111+0000","last_unstale":"2023-11-28T11:53:53.367391+0000","last_undegraded":"2023-11-28T11:53:53.367391+0000","last_fullsized":"2023-11-28T11:53:53.367391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:58.872124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366648+0000","last_change":"2023-11-28T11:53:53.364232+0000","last_active":"2023-11-28T11:53:53.366648+0000","last_peered":"2023-11-28T11:53:53.366648+0000","last_clean":"2023-11-28T11:53:53.366648+0000","last_became_active":"2023-11-28T11:53:53.363973+0000","last_became_peered":"2023-11-28T11:53:53.363973+0000","last_unstale":"2023-11-28T11:53:53.366648+0000","last_undegraded":"2023-11-28T11:53:53.366648+0000","last_fullsized":"2023-11-28T11:53:53.366648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:27.701979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348854+0000","last_change":"2023-11-28T11:53:53.364319+0000","last_active":"2023-11-28T11:53:54.348854+0000","last_peered":"2023-11-28T11:53:54.348854+0000","last_clean":"2023-11-28T11:53:54.348854+0000","last_became_active":"2023-11-28T11:53:53.363933+0000","last_became_peered":"2023-11-28T11:53:53.363933+0000","last_unstale":"2023-11-28T11:53:54.348854+0000","last_undegraded":"2023-11-28T11:53:54.348854+0000","last_fullsized":"2023-11-28T11:53:54.348854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:27.701979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.371686+0000","last_change":"2023-11-28T11:53:53.368196+0000","last_active":"2023-11-28T11:53:53.371686+0000","last_peered":"2023-11-28T11:53:53.371686+0000","last_clean":"2023-11-28T11:53:53.371686+0000","last_became_active":"2023-11-28T11:53:53.367915+0000","last_became_peered":"2023-11-28T11:53:53.367915+0000","last_unstale":"2023-11-28T11:53:53.371686+0000","last_undegraded":"2023-11-28T11:53:53.371686+0000","last_fullsized":"2023-11-28T11:53:53.371686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:19.117644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823332+0000","last_change":"2023-11-28T11:53:53.366020+0000","last_active":"2023-11-28T11:53:54.823332+0000","last_peered":"2023-11-28T11:53:54.823332+0000","last_clean":"2023-11-28T11:53:54.823332+0000","last_became_active":"2023-11-28T11:53:53.365791+0000","last_became_peered":"2023-11-28T11:53:53.365791+0000","last_unstale":"2023-11-28T11:53:54.823332+0000","last_undegraded":"2023-11-28T11:53:54.823332+0000","last_fullsized":"2023-11-28T11:53:54.823332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:33.277898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.362502+0000","last_change":"2023-11-28T11:53:53.360962+0000","last_active":"2023-11-28T11:53:53.362502+0000","last_peered":"2023-11-28T11:53:53.362502+0000","last_clean":"2023-11-28T11:53:53.362502+0000","last_became_active":"2023-11-28T11:53:53.360718+0000","last_became_peered":"2023-11-28T11:53:53.360718+0000","last_unstale":"2023-11-28T11:53:53.362502+0000","last_undegraded":"2023-11-28T11:53:53.362502+0000","last_fullsized":"2023-11-28T11:53:53.362502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:41.319939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347331+0000","last_change":"2023-11-28T11:53:53.358618+0000","last_active":"2023-11-28T11:53:54.347331+0000","last_peered":"2023-11-28T11:53:54.347331+0000","last_clean":"2023-11-28T11:53:54.347331+0000","last_became_active":"2023-11-28T11:53:53.358351+0000","last_became_peered":"2023-11-28T11:53:53.358351+0000","last_unstale":"2023-11-28T11:53:54.347331+0000","last_undegraded":"2023-11-28T11:53:54.347331+0000","last_fullsized":"2023-11-28T11:53:54.347331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:28:57.572773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366694+0000","last_change":"2023-11-28T11:53:53.365311+0000","last_active":"2023-11-28T11:53:53.366694+0000","last_peered":"2023-11-28T11:53:53.366694+0000","last_clean":"2023-11-28T11:53:53.366694+0000","last_became_active":"2023-11-28T11:53:53.364487+0000","last_became_peered":"2023-11-28T11:53:53.364487+0000","last_unstale":"2023-11-28T11:53:53.366694+0000","last_undegraded":"2023-11-28T11:53:53.366694+0000","last_fullsized":"2023-11-28T11:53:53.366694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:17:18.508153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.360514+0000","last_change":"2023-11-28T11:53:53.357892+0000","last_active":"2023-11-28T11:53:53.360514+0000","last_peered":"2023-11-28T11:53:53.360514+0000","last_clean":"2023-11-28T11:53:53.360514+0000","last_became_active":"2023-11-28T11:53:53.356964+0000","last_became_peered":"2023-11-28T11:53:53.356964+0000","last_unstale":"2023-11-28T11:53:53.360514+0000","last_undegraded":"2023-11-28T11:53:53.360514+0000","last_fullsized":"2023-11-28T11:53:53.360514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:30.442863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348510+0000","last_change":"2023-11-28T11:53:53.365848+0000","last_active":"2023-11-28T11:53:54.348510+0000","last_peered":"2023-11-28T11:53:54.348510+0000","last_clean":"2023-11-28T11:53:54.348510+0000","last_became_active":"2023-11-28T11:53:53.365045+0000","last_became_peered":"2023-11-28T11:53:53.365045+0000","last_unstale":"2023-11-28T11:53:54.348510+0000","last_undegraded":"2023-11-28T11:53:54.348510+0000","last_fullsized":"2023-11-28T11:53:54.348510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:18.360820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347448+0000","last_change":"2023-11-28T11:53:53.358033+0000","last_active":"2023-11-28T11:53:54.347448+0000","last_peered":"2023-11-28T11:53:54.347448+0000","last_clean":"2023-11-28T11:53:54.347448+0000","last_became_active":"2023-11-28T11:53:53.357727+0000","last_became_peered":"2023-11-28T11:53:53.357727+0000","last_unstale":"2023-11-28T11:53:54.347448+0000","last_undegraded":"2023-11-28T11:53:54.347448+0000","last_fullsized":"2023-11-28T11:53:54.347448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:42.836331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347520+0000","last_change":"2023-11-28T11:53:53.367259+0000","last_active":"2023-11-28T11:53:54.347520+0000","last_peered":"2023-11-28T11:53:54.347520+0000","last_clean":"2023-11-28T11:53:54.347520+0000","last_became_active":"2023-11-28T11:53:53.366982+0000","last_became_peered":"2023-11-28T11:53:53.366982+0000","last_unstale":"2023-11-28T11:53:54.347520+0000","last_undegraded":"2023-11-28T11:53:54.347520+0000","last_fullsized":"2023-11-28T11:53:54.347520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:28.183193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366084+0000","last_change":"2023-11-28T11:53:53.361963+0000","last_active":"2023-11-28T11:53:53.366084+0000","last_peered":"2023-11-28T11:53:53.366084+0000","last_clean":"2023-11-28T11:53:53.366084+0000","last_became_active":"2023-11-28T11:53:53.361656+0000","last_became_peered":"2023-11-28T11:53:53.361656+0000","last_unstale":"2023-11-28T11:53:53.366084+0000","last_undegraded":"2023-11-28T11:53:53.366084+0000","last_fullsized":"2023-11-28T11:53:53.366084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:20.156694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363018+0000","last_change":"2023-11-28T11:53:53.359500+0000","last_active":"2023-11-28T11:53:53.363018+0000","last_peered":"2023-11-28T11:53:53.363018+0000","last_clean":"2023-11-28T11:53:53.363018+0000","last_became_active":"2023-11-28T11:53:53.354835+0000","last_became_peered":"2023-11-28T11:53:53.354835+0000","last_unstale":"2023-11-28T11:53:53.363018+0000","last_undegraded":"2023-11-28T11:53:53.363018+0000","last_fullsized":"2023-11-28T11:53:53.363018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:42.420815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363619+0000","last_change":"2023-11-28T11:53:53.352510+0000","last_active":"2023-11-28T11:53:53.363619+0000","last_peered":"2023-11-28T11:53:53.363619+0000","last_clean":"2023-11-28T11:53:53.363619+0000","last_became_active":"2023-11-28T11:53:53.352277+0000","last_became_peered":"2023-11-28T11:53:53.352277+0000","last_unstale":"2023-11-28T11:53:53.363619+0000","last_undegraded":"2023-11-28T11:53:53.363619+0000","last_fullsized":"2023-11-28T11:53:53.363619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:45:59.032830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.369524+0000","last_change":"2023-11-28T11:53:53.368953+0000","last_active":"2023-11-28T11:53:53.369524+0000","last_peered":"2023-11-28T11:53:53.369524+0000","last_clean":"2023-11-28T11:53:53.369524+0000","last_became_active":"2023-11-28T11:53:53.368752+0000","last_became_peered":"2023-11-28T11:53:53.368752+0000","last_unstale":"2023-11-28T11:53:53.369524+0000","last_undegraded":"2023-11-28T11:53:53.369524+0000","last_fullsized":"2023-11-28T11:53:53.369524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:06.472324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348524+0000","last_change":"2023-11-28T11:53:53.363865+0000","last_active":"2023-11-28T11:53:54.348524+0000","last_peered":"2023-11-28T11:53:54.348524+0000","last_clean":"2023-11-28T11:53:54.348524+0000","last_became_active":"2023-11-28T11:53:53.363551+0000","last_became_peered":"2023-11-28T11:53:53.363551+0000","last_unstale":"2023-11-28T11:53:54.348524+0000","last_undegraded":"2023-11-28T11:53:54.348524+0000","last_fullsized":"2023-11-28T11:53:54.348524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:37.461966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357711+0000","last_change":"2023-11-28T11:53:53.352764+0000","last_active":"2023-11-28T11:53:53.357711+0000","last_peered":"2023-11-28T11:53:53.357711+0000","last_clean":"2023-11-28T11:53:53.357711+0000","last_became_active":"2023-11-28T11:53:53.352491+0000","last_became_peered":"2023-11-28T11:53:53.352491+0000","last_unstale":"2023-11-28T11:53:53.357711+0000","last_undegraded":"2023-11-28T11:53:53.357711+0000","last_fullsized":"2023-11-28T11:53:53.357711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:58.872124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347708+0000","last_change":"2023-11-28T11:53:53.349348+0000","last_active":"2023-11-28T11:53:54.347708+0000","last_peered":"2023-11-28T11:53:54.347708+0000","last_clean":"2023-11-28T11:53:54.347708+0000","last_became_active":"2023-11-28T11:53:53.349086+0000","last_became_peered":"2023-11-28T11:53:53.349086+0000","last_unstale":"2023-11-28T11:53:54.347708+0000","last_undegraded":"2023-11-28T11:53:54.347708+0000","last_fullsized":"2023-11-28T11:53:54.347708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:12.565467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347631+0000","last_change":"2023-11-28T11:53:53.355325+0000","last_active":"2023-11-28T11:53:54.347631+0000","last_peered":"2023-11-28T11:53:54.347631+0000","last_clean":"2023-11-28T11:53:54.347631+0000","last_became_active":"2023-11-28T11:53:53.355066+0000","last_became_peered":"2023-11-28T11:53:53.355066+0000","last_unstale":"2023-11-28T11:53:54.347631+0000","last_undegraded":"2023-11-28T11:53:54.347631+0000","last_fullsized":"2023-11-28T11:53:54.347631+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:20.156694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347638+0000","last_change":"2023-11-28T11:53:53.357058+0000","last_active":"2023-11-28T11:53:54.347638+0000","last_peered":"2023-11-28T11:53:54.347638+0000","last_clean":"2023-11-28T11:53:54.347638+0000","last_became_active":"2023-11-28T11:53:53.352948+0000","last_became_peered":"2023-11-28T11:53:53.352948+0000","last_unstale":"2023-11-28T11:53:54.347638+0000","last_undegraded":"2023-11-28T11:53:54.347638+0000","last_fullsized":"2023-11-28T11:53:54.347638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:26.522177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347576+0000","last_change":"2023-11-28T11:53:53.356684+0000","last_active":"2023-11-28T11:53:54.347576+0000","last_peered":"2023-11-28T11:53:54.347576+0000","last_clean":"2023-11-28T11:53:54.347576+0000","last_became_active":"2023-11-28T11:53:53.352988+0000","last_became_peered":"2023-11-28T11:53:53.352988+0000","last_unstale":"2023-11-28T11:53:54.347576+0000","last_undegraded":"2023-11-28T11:53:54.347576+0000","last_fullsized":"2023-11-28T11:53:54.347576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:10.953057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365866+0000","last_change":"2023-11-28T11:53:53.362055+0000","last_active":"2023-11-28T11:53:53.365866+0000","last_peered":"2023-11-28T11:53:53.365866+0000","last_clean":"2023-11-28T11:53:53.365866+0000","last_became_active":"2023-11-28T11:53:53.361677+0000","last_became_peered":"2023-11-28T11:53:53.361677+0000","last_unstale":"2023-11-28T11:53:53.365866+0000","last_undegraded":"2023-11-28T11:53:53.365866+0000","last_fullsized":"2023-11-28T11:53:53.365866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:41.839130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348907+0000","last_change":"2023-11-28T11:53:53.362185+0000","last_active":"2023-11-28T11:53:54.348907+0000","last_peered":"2023-11-28T11:53:54.348907+0000","last_clean":"2023-11-28T11:53:54.348907+0000","last_became_active":"2023-11-28T11:53:53.360150+0000","last_became_peered":"2023-11-28T11:53:53.360150+0000","last_unstale":"2023-11-28T11:53:54.348907+0000","last_undegraded":"2023-11-28T11:53:54.348907+0000","last_fullsized":"2023-11-28T11:53:54.348907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:20.242319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347947+0000","last_change":"2023-11-28T11:53:53.354785+0000","last_active":"2023-11-28T11:53:54.347947+0000","last_peered":"2023-11-28T11:53:54.347947+0000","last_clean":"2023-11-28T11:53:54.347947+0000","last_became_active":"2023-11-28T11:53:53.354519+0000","last_became_peered":"2023-11-28T11:53:53.354519+0000","last_unstale":"2023-11-28T11:53:54.347947+0000","last_undegraded":"2023-11-28T11:53:54.347947+0000","last_fullsized":"2023-11-28T11:53:54.347947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:17:18.508153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348557+0000","last_change":"2023-11-28T11:53:53.366035+0000","last_active":"2023-11-28T11:53:54.348557+0000","last_peered":"2023-11-28T11:53:54.348557+0000","last_clean":"2023-11-28T11:53:54.348557+0000","last_became_active":"2023-11-28T11:53:53.365383+0000","last_became_peered":"2023-11-28T11:53:53.365383+0000","last_unstale":"2023-11-28T11:53:54.348557+0000","last_undegraded":"2023-11-28T11:53:54.348557+0000","last_fullsized":"2023-11-28T11:53:54.348557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:42.420815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365520+0000","last_change":"2023-11-28T11:53:53.360229+0000","last_active":"2023-11-28T11:53:53.365520+0000","last_peered":"2023-11-28T11:53:53.365520+0000","last_clean":"2023-11-28T11:53:53.365520+0000","last_became_active":"2023-11-28T11:53:53.359636+0000","last_became_peered":"2023-11-28T11:53:53.359636+0000","last_unstale":"2023-11-28T11:53:53.365520+0000","last_undegraded":"2023-11-28T11:53:53.365520+0000","last_fullsized":"2023-11-28T11:53:53.365520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:44.146115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.371233+0000","last_change":"2023-11-28T11:53:53.367844+0000","last_active":"2023-11-28T11:53:53.371233+0000","last_peered":"2023-11-28T11:53:53.371233+0000","last_clean":"2023-11-28T11:53:53.371233+0000","last_became_active":"2023-11-28T11:53:53.367601+0000","last_became_peered":"2023-11-28T11:53:53.367601+0000","last_unstale":"2023-11-28T11:53:53.371233+0000","last_undegraded":"2023-11-28T11:53:53.371233+0000","last_fullsized":"2023-11-28T11:53:53.371233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:58.872124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347590+0000","last_change":"2023-11-28T11:53:53.357945+0000","last_active":"2023-11-28T11:53:54.347590+0000","last_peered":"2023-11-28T11:53:54.347590+0000","last_clean":"2023-11-28T11:53:54.347590+0000","last_became_active":"2023-11-28T11:53:53.357565+0000","last_became_peered":"2023-11-28T11:53:53.357565+0000","last_unstale":"2023-11-28T11:53:54.347590+0000","last_undegraded":"2023-11-28T11:53:54.347590+0000","last_fullsized":"2023-11-28T11:53:54.347590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:35:00.695698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348343+0000","last_change":"2023-11-28T11:53:53.365350+0000","last_active":"2023-11-28T11:53:54.348343+0000","last_peered":"2023-11-28T11:53:54.348343+0000","last_clean":"2023-11-28T11:53:54.348343+0000","last_became_active":"2023-11-28T11:53:53.365027+0000","last_became_peered":"2023-11-28T11:53:53.365027+0000","last_unstale":"2023-11-28T11:53:54.348343+0000","last_undegraded":"2023-11-28T11:53:54.348343+0000","last_fullsized":"2023-11-28T11:53:54.348343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:32:20.156694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823268+0000","last_change":"2023-11-28T11:53:53.357753+0000","last_active":"2023-11-28T11:53:54.823268+0000","last_peered":"2023-11-28T11:53:54.823268+0000","last_clean":"2023-11-28T11:53:54.823268+0000","last_became_active":"2023-11-28T11:53:53.357354+0000","last_became_peered":"2023-11-28T11:53:53.357354+0000","last_unstale":"2023-11-28T11:53:54.823268+0000","last_undegraded":"2023-11-28T11:53:54.823268+0000","last_fullsized":"2023-11-28T11:53:54.823268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:57:27.118800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.360469+0000","last_change":"2023-11-28T11:53:53.354154+0000","last_active":"2023-11-28T11:53:53.360469+0000","last_peered":"2023-11-28T11:53:53.360469+0000","last_clean":"2023-11-28T11:53:53.360469+0000","last_became_active":"2023-11-28T11:53:53.353843+0000","last_became_peered":"2023-11-28T11:53:53.353843+0000","last_unstale":"2023-11-28T11:53:53.360469+0000","last_undegraded":"2023-11-28T11:53:53.360469+0000","last_fullsized":"2023-11-28T11:53:53.360469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:57.062194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"59'4","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.369164+0000","last_change":"2023-11-28T11:53:52.940645+0000","last_active":"2023-11-28T11:53:53.369164+0000","last_peered":"2023-11-28T11:53:53.369164+0000","last_clean":"2023-11-28T11:53:53.369164+0000","last_became_active":"2023-11-28T11:53:52.940278+0000","last_became_peered":"2023-11-28T11:53:52.940278+0000","last_unstale":"2023-11-28T11:53:53.369164+0000","last_undegraded":"2023-11-28T11:53:53.369164+0000","last_fullsized":"2023-11-28T11:53:53.369164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:43.982633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348632+0000","last_change":"2023-11-28T11:53:53.365669+0000","last_active":"2023-11-28T11:53:54.348632+0000","last_peered":"2023-11-28T11:53:54.348632+0000","last_clean":"2023-11-28T11:53:54.348632+0000","last_became_active":"2023-11-28T11:53:53.364781+0000","last_became_peered":"2023-11-28T11:53:53.364781+0000","last_unstale":"2023-11-28T11:53:54.348632+0000","last_undegraded":"2023-11-28T11:53:54.348632+0000","last_fullsized":"2023-11-28T11:53:54.348632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:41.839130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823663+0000","last_change":"2023-11-28T11:53:52.948760+0000","last_active":"2023-11-28T11:53:54.823663+0000","last_peered":"2023-11-28T11:53:54.823663+0000","last_clean":"2023-11-28T11:53:54.823663+0000","last_became_active":"2023-11-28T11:53:52.948469+0000","last_became_peered":"2023-11-28T11:53:52.948469+0000","last_unstale":"2023-11-28T11:53:54.823663+0000","last_undegraded":"2023-11-28T11:53:54.823663+0000","last_fullsized":"2023-11-28T11:53:54.823663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:55.310728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823648+0000","last_change":"2023-11-28T11:53:53.362852+0000","last_active":"2023-11-28T11:53:54.823648+0000","last_peered":"2023-11-28T11:53:54.823648+0000","last_clean":"2023-11-28T11:53:54.823648+0000","last_became_active":"2023-11-28T11:53:53.362618+0000","last_became_peered":"2023-11-28T11:53:53.362618+0000","last_unstale":"2023-11-28T11:53:54.823648+0000","last_undegraded":"2023-11-28T11:53:54.823648+0000","last_fullsized":"2023-11-28T11:53:54.823648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:33.067440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823622+0000","last_change":"2023-11-28T11:53:52.947525+0000","last_active":"2023-11-28T11:53:54.823622+0000","last_peered":"2023-11-28T11:53:54.823622+0000","last_clean":"2023-11-28T11:53:54.823622+0000","last_became_active":"2023-11-28T11:53:52.946843+0000","last_became_peered":"2023-11-28T11:53:52.946843+0000","last_unstale":"2023-11-28T11:53:54.823622+0000","last_undegraded":"2023-11-28T11:53:54.823622+0000","last_fullsized":"2023-11-28T11:53:54.823622+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:30.139541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.358477+0000","last_change":"2023-11-28T11:53:53.356387+0000","last_active":"2023-11-28T11:53:53.358477+0000","last_peered":"2023-11-28T11:53:53.358477+0000","last_clean":"2023-11-28T11:53:53.358477+0000","last_became_active":"2023-11-28T11:53:53.355819+0000","last_became_peered":"2023-11-28T11:53:53.355819+0000","last_unstale":"2023-11-28T11:53:53.358477+0000","last_undegraded":"2023-11-28T11:53:53.358477+0000","last_fullsized":"2023-11-28T11:53:53.358477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:28.183193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823542+0000","last_change":"2023-11-28T11:53:52.943916+0000","last_active":"2023-11-28T11:53:54.823542+0000","last_peered":"2023-11-28T11:53:54.823542+0000","last_clean":"2023-11-28T11:53:54.823542+0000","last_became_active":"2023-11-28T11:53:52.943685+0000","last_became_peered":"2023-11-28T11:53:52.943685+0000","last_unstale":"2023-11-28T11:53:54.823542+0000","last_undegraded":"2023-11-28T11:53:54.823542+0000","last_fullsized":"2023-11-28T11:53:54.823542+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:17.286301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363641+0000","last_change":"2023-11-28T11:53:53.353957+0000","last_active":"2023-11-28T11:53:53.363641+0000","last_peered":"2023-11-28T11:53:53.363641+0000","last_clean":"2023-11-28T11:53:53.363641+0000","last_became_active":"2023-11-28T11:53:53.353720+0000","last_became_peered":"2023-11-28T11:53:53.353720+0000","last_unstale":"2023-11-28T11:53:53.363641+0000","last_undegraded":"2023-11-28T11:53:53.363641+0000","last_fullsized":"2023-11-28T11:53:53.363641+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:17.297085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347800+0000","last_change":"2023-11-28T11:53:52.944810+0000","last_active":"2023-11-28T11:53:54.347800+0000","last_peered":"2023-11-28T11:53:54.347800+0000","last_clean":"2023-11-28T11:53:54.347800+0000","last_became_active":"2023-11-28T11:53:52.944485+0000","last_became_peered":"2023-11-28T11:53:52.944485+0000","last_unstale":"2023-11-28T11:53:54.347800+0000","last_undegraded":"2023-11-28T11:53:54.347800+0000","last_fullsized":"2023-11-28T11:53:54.347800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:42.354305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367411+0000","last_change":"2023-11-28T11:53:53.365879+0000","last_active":"2023-11-28T11:53:53.367411+0000","last_peered":"2023-11-28T11:53:53.367411+0000","last_clean":"2023-11-28T11:53:53.367411+0000","last_became_active":"2023-11-28T11:53:53.365663+0000","last_became_peered":"2023-11-28T11:53:53.365663+0000","last_unstale":"2023-11-28T11:53:53.367411+0000","last_undegraded":"2023-11-28T11:53:53.367411+0000","last_fullsized":"2023-11-28T11:53:53.367411+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.844830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347624+0000","last_change":"2023-11-28T11:53:52.944839+0000","last_active":"2023-11-28T11:53:54.347624+0000","last_peered":"2023-11-28T11:53:54.347624+0000","last_clean":"2023-11-28T11:53:54.347624+0000","last_became_active":"2023-11-28T11:53:52.944553+0000","last_became_peered":"2023-11-28T11:53:52.944553+0000","last_unstale":"2023-11-28T11:53:54.347624+0000","last_undegraded":"2023-11-28T11:53:54.347624+0000","last_fullsized":"2023-11-28T11:53:54.347624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:33:14.910583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363430+0000","last_change":"2023-11-28T11:53:53.359147+0000","last_active":"2023-11-28T11:53:53.363430+0000","last_peered":"2023-11-28T11:53:53.363430+0000","last_clean":"2023-11-28T11:53:53.363430+0000","last_became_active":"2023-11-28T11:53:53.358810+0000","last_became_peered":"2023-11-28T11:53:53.358810+0000","last_unstale":"2023-11-28T11:53:53.363430+0000","last_undegraded":"2023-11-28T11:53:53.363430+0000","last_fullsized":"2023-11-28T11:53:53.363430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:32.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347662+0000","last_change":"2023-11-28T11:53:52.942091+0000","last_active":"2023-11-28T11:53:54.347662+0000","last_peered":"2023-11-28T11:53:54.347662+0000","last_clean":"2023-11-28T11:53:54.347662+0000","last_became_active":"2023-11-28T11:53:52.941291+0000","last_became_peered":"2023-11-28T11:53:52.941291+0000","last_unstale":"2023-11-28T11:53:54.347662+0000","last_undegraded":"2023-11-28T11:53:54.347662+0000","last_fullsized":"2023-11-28T11:53:54.347662+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:59.693541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347718+0000","last_change":"2023-11-28T11:53:53.367244+0000","last_active":"2023-11-28T11:53:54.347718+0000","last_peered":"2023-11-28T11:53:54.347718+0000","last_clean":"2023-11-28T11:53:54.347718+0000","last_became_active":"2023-11-28T11:53:53.366967+0000","last_became_peered":"2023-11-28T11:53:53.366967+0000","last_unstale":"2023-11-28T11:53:54.347718+0000","last_undegraded":"2023-11-28T11:53:54.347718+0000","last_fullsized":"2023-11-28T11:53:54.347718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:36:17.121804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.343023+0000","last_change":"2023-11-28T11:53:52.947769+0000","last_active":"2023-11-28T11:53:53.343023+0000","last_peered":"2023-11-28T11:53:53.343023+0000","last_clean":"2023-11-28T11:53:53.343023+0000","last_became_active":"2023-11-28T11:53:52.947502+0000","last_became_peered":"2023-11-28T11:53:52.947502+0000","last_unstale":"2023-11-28T11:53:53.343023+0000","last_undegraded":"2023-11-28T11:53:53.343023+0000","last_fullsized":"2023-11-28T11:53:53.343023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:29.907062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.361746+0000","last_change":"2023-11-28T11:53:53.353843+0000","last_active":"2023-11-28T11:53:53.361746+0000","last_peered":"2023-11-28T11:53:53.361746+0000","last_clean":"2023-11-28T11:53:53.361746+0000","last_became_active":"2023-11-28T11:53:53.353553+0000","last_became_peered":"2023-11-28T11:53:53.353553+0000","last_unstale":"2023-11-28T11:53:53.361746+0000","last_undegraded":"2023-11-28T11:53:53.361746+0000","last_fullsized":"2023-11-28T11:53:53.361746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:32.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.346649+0000","last_change":"2023-11-28T11:53:52.949620+0000","last_active":"2023-11-28T11:53:53.346649+0000","last_peered":"2023-11-28T11:53:53.346649+0000","last_clean":"2023-11-28T11:53:53.346649+0000","last_became_active":"2023-11-28T11:53:52.949361+0000","last_became_peered":"2023-11-28T11:53:52.949361+0000","last_unstale":"2023-11-28T11:53:53.346649+0000","last_undegraded":"2023-11-28T11:53:53.346649+0000","last_fullsized":"2023-11-28T11:53:53.346649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:59.950858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347793+0000","last_change":"2023-11-28T11:53:53.350460+0000","last_active":"2023-11-28T11:53:54.347793+0000","last_peered":"2023-11-28T11:53:54.347793+0000","last_clean":"2023-11-28T11:53:54.347793+0000","last_became_active":"2023-11-28T11:53:53.350200+0000","last_became_peered":"2023-11-28T11:53:53.350200+0000","last_unstale":"2023-11-28T11:53:54.347793+0000","last_undegraded":"2023-11-28T11:53:54.347793+0000","last_fullsized":"2023-11-28T11:53:54.347793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:01.087022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348736+0000","last_change":"2023-11-28T11:53:52.937666+0000","last_active":"2023-11-28T11:53:54.348736+0000","last_peered":"2023-11-28T11:53:54.348736+0000","last_clean":"2023-11-28T11:53:54.348736+0000","last_became_active":"2023-11-28T11:53:52.937316+0000","last_became_peered":"2023-11-28T11:53:52.937316+0000","last_unstale":"2023-11-28T11:53:54.348736+0000","last_undegraded":"2023-11-28T11:53:54.348736+0000","last_fullsized":"2023-11-28T11:53:54.348736+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:35.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357174+0000","last_change":"2023-11-28T11:53:53.350812+0000","last_active":"2023-11-28T11:53:53.357174+0000","last_peered":"2023-11-28T11:53:53.357174+0000","last_clean":"2023-11-28T11:53:53.357174+0000","last_became_active":"2023-11-28T11:53:53.350172+0000","last_became_peered":"2023-11-28T11:53:53.350172+0000","last_unstale":"2023-11-28T11:53:53.357174+0000","last_undegraded":"2023-11-28T11:53:53.357174+0000","last_fullsized":"2023-11-28T11:53:53.357174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:01.100398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"59'3","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365831+0000","last_change":"2023-11-28T11:53:52.949523+0000","last_active":"2023-11-28T11:53:53.365831+0000","last_peered":"2023-11-28T11:53:53.365831+0000","last_clean":"2023-11-28T11:53:53.365831+0000","last_became_active":"2023-11-28T11:53:52.949294+0000","last_became_peered":"2023-11-28T11:53:52.949294+0000","last_unstale":"2023-11-28T11:53:53.365831+0000","last_undegraded":"2023-11-28T11:53:53.365831+0000","last_fullsized":"2023-11-28T11:53:53.365831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:56.060037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348425+0000","last_change":"2023-11-28T11:53:53.363410+0000","last_active":"2023-11-28T11:53:54.348425+0000","last_peered":"2023-11-28T11:53:54.348425+0000","last_clean":"2023-11-28T11:53:54.348425+0000","last_became_active":"2023-11-28T11:53:53.363127+0000","last_became_peered":"2023-11-28T11:53:53.363127+0000","last_unstale":"2023-11-28T11:53:54.348425+0000","last_undegraded":"2023-11-28T11:53:54.348425+0000","last_fullsized":"2023-11-28T11:53:54.348425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:48.369328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"59'6","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.374665+0000","last_change":"2023-11-28T11:53:52.941884+0000","last_active":"2023-11-28T11:53:53.374665+0000","last_peered":"2023-11-28T11:53:53.374665+0000","last_clean":"2023-11-28T11:53:53.374665+0000","last_became_active":"2023-11-28T11:53:52.941446+0000","last_became_peered":"2023-11-28T11:53:52.941446+0000","last_unstale":"2023-11-28T11:53:53.374665+0000","last_undegraded":"2023-11-28T11:53:53.374665+0000","last_fullsized":"2023-11-28T11:53:53.374665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:38:50.274987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348036+0000","last_change":"2023-11-28T11:53:53.363425+0000","last_active":"2023-11-28T11:53:54.348036+0000","last_peered":"2023-11-28T11:53:54.348036+0000","last_clean":"2023-11-28T11:53:54.348036+0000","last_became_active":"2023-11-28T11:53:53.363118+0000","last_became_peered":"2023-11-28T11:53:53.363118+0000","last_unstale":"2023-11-28T11:53:54.348036+0000","last_undegraded":"2023-11-28T11:53:54.348036+0000","last_fullsized":"2023-11-28T11:53:54.348036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:58.402661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.364230+0000","last_change":"2023-11-28T11:53:52.947103+0000","last_active":"2023-11-28T11:53:53.364230+0000","last_peered":"2023-11-28T11:53:53.364230+0000","last_clean":"2023-11-28T11:53:53.364230+0000","last_became_active":"2023-11-28T11:53:52.946805+0000","last_became_peered":"2023-11-28T11:53:52.946805+0000","last_unstale":"2023-11-28T11:53:53.364230+0000","last_undegraded":"2023-11-28T11:53:53.364230+0000","last_fullsized":"2023-11-28T11:53:53.364230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:37.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347328+0000","last_change":"2023-11-28T11:53:53.357625+0000","last_active":"2023-11-28T11:53:54.347328+0000","last_peered":"2023-11-28T11:53:54.347328+0000","last_clean":"2023-11-28T11:53:54.347328+0000","last_became_active":"2023-11-28T11:53:53.357330+0000","last_became_peered":"2023-11-28T11:53:53.357330+0000","last_unstale":"2023-11-28T11:53:54.347328+0000","last_undegraded":"2023-11-28T11:53:54.347328+0000","last_fullsized":"2023-11-28T11:53:54.347328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:37.461966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347683+0000","last_change":"2023-11-28T11:53:52.946327+0000","last_active":"2023-11-28T11:53:54.347683+0000","last_peered":"2023-11-28T11:53:54.347683+0000","last_clean":"2023-11-28T11:53:54.347683+0000","last_became_active":"2023-11-28T11:53:52.946031+0000","last_became_peered":"2023-11-28T11:53:52.946031+0000","last_unstale":"2023-11-28T11:53:54.347683+0000","last_undegraded":"2023-11-28T11:53:54.347683+0000","last_fullsized":"2023-11-28T11:53:54.347683+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:31.298250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357675+0000","last_change":"2023-11-28T11:53:53.352629+0000","last_active":"2023-11-28T11:53:53.357675+0000","last_peered":"2023-11-28T11:53:53.357675+0000","last_clean":"2023-11-28T11:53:53.357675+0000","last_became_active":"2023-11-28T11:53:53.352392+0000","last_became_peered":"2023-11-28T11:53:53.352392+0000","last_unstale":"2023-11-28T11:53:53.357675+0000","last_undegraded":"2023-11-28T11:53:53.357675+0000","last_fullsized":"2023-11-28T11:53:53.357675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:14:44.146115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.346091+0000","last_change":"2023-11-28T11:53:52.945727+0000","last_active":"2023-11-28T11:53:53.346091+0000","last_peered":"2023-11-28T11:53:53.346091+0000","last_clean":"2023-11-28T11:53:53.346091+0000","last_became_active":"2023-11-28T11:53:52.945394+0000","last_became_peered":"2023-11-28T11:53:52.945394+0000","last_unstale":"2023-11-28T11:53:53.346091+0000","last_undegraded":"2023-11-28T11:53:53.346091+0000","last_fullsized":"2023-11-28T11:53:53.346091+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:20.339134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.364921+0000","last_change":"2023-11-28T11:53:53.360149+0000","last_active":"2023-11-28T11:53:53.364921+0000","last_peered":"2023-11-28T11:53:53.364921+0000","last_clean":"2023-11-28T11:53:53.364921+0000","last_became_active":"2023-11-28T11:53:53.359902+0000","last_became_peered":"2023-11-28T11:53:53.359902+0000","last_unstale":"2023-11-28T11:53:53.364921+0000","last_undegraded":"2023-11-28T11:53:53.364921+0000","last_fullsized":"2023-11-28T11:53:53.364921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:48.369328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"59'8","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348058+0000","last_change":"2023-11-28T11:53:52.940538+0000","last_active":"2023-11-28T11:53:54.348058+0000","last_peered":"2023-11-28T11:53:54.348058+0000","last_clean":"2023-11-28T11:53:54.348058+0000","last_became_active":"2023-11-28T11:53:52.940228+0000","last_became_peered":"2023-11-28T11:53:52.940228+0000","last_unstale":"2023-11-28T11:53:54.348058+0000","last_undegraded":"2023-11-28T11:53:54.348058+0000","last_fullsized":"2023-11-28T11:53:54.348058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:10.424141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823547+0000","last_change":"2023-11-28T11:53:53.367540+0000","last_active":"2023-11-28T11:53:54.823547+0000","last_peered":"2023-11-28T11:53:54.823547+0000","last_clean":"2023-11-28T11:53:54.823547+0000","last_became_active":"2023-11-28T11:53:53.367276+0000","last_became_peered":"2023-11-28T11:53:53.367276+0000","last_unstale":"2023-11-28T11:53:54.823547+0000","last_undegraded":"2023-11-28T11:53:54.823547+0000","last_fullsized":"2023-11-28T11:53:54.823547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:25:42.073775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348942+0000","last_change":"2023-11-28T11:53:52.947823+0000","last_active":"2023-11-28T11:53:54.348942+0000","last_peered":"2023-11-28T11:53:54.348942+0000","last_clean":"2023-11-28T11:53:54.348942+0000","last_became_active":"2023-11-28T11:53:52.947513+0000","last_became_peered":"2023-11-28T11:53:52.947513+0000","last_unstale":"2023-11-28T11:53:54.348942+0000","last_undegraded":"2023-11-28T11:53:54.348942+0000","last_fullsized":"2023-11-28T11:53:54.348942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:20.339134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.362443+0000","last_change":"2023-11-28T11:53:53.357944+0000","last_active":"2023-11-28T11:53:53.362443+0000","last_peered":"2023-11-28T11:53:53.362443+0000","last_clean":"2023-11-28T11:53:53.362443+0000","last_became_active":"2023-11-28T11:53:53.357630+0000","last_became_peered":"2023-11-28T11:53:53.357630+0000","last_unstale":"2023-11-28T11:53:53.362443+0000","last_undegraded":"2023-11-28T11:53:53.362443+0000","last_fullsized":"2023-11-28T11:53:53.362443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:22:59.644754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348118+0000","last_change":"2023-11-28T11:53:52.947074+0000","last_active":"2023-11-28T11:53:54.348118+0000","last_peered":"2023-11-28T11:53:54.348118+0000","last_clean":"2023-11-28T11:53:54.348118+0000","last_became_active":"2023-11-28T11:53:52.946804+0000","last_became_peered":"2023-11-28T11:53:52.946804+0000","last_unstale":"2023-11-28T11:53:54.348118+0000","last_undegraded":"2023-11-28T11:53:54.348118+0000","last_fullsized":"2023-11-28T11:53:54.348118+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:59.950858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348040+0000","last_change":"2023-11-28T11:53:53.362038+0000","last_active":"2023-11-28T11:53:54.348040+0000","last_peered":"2023-11-28T11:53:54.348040+0000","last_clean":"2023-11-28T11:53:54.348040+0000","last_became_active":"2023-11-28T11:53:53.361428+0000","last_became_peered":"2023-11-28T11:53:53.361428+0000","last_unstale":"2023-11-28T11:53:54.348040+0000","last_undegraded":"2023-11-28T11:53:54.348040+0000","last_fullsized":"2023-11-28T11:53:54.348040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:10:30.442863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823652+0000","last_change":"2023-11-28T11:53:52.945655+0000","last_active":"2023-11-28T11:53:54.823652+0000","last_peered":"2023-11-28T11:53:54.823652+0000","last_clean":"2023-11-28T11:53:54.823652+0000","last_became_active":"2023-11-28T11:53:52.945344+0000","last_became_peered":"2023-11-28T11:53:52.945344+0000","last_unstale":"2023-11-28T11:53:54.823652+0000","last_undegraded":"2023-11-28T11:53:54.823652+0000","last_fullsized":"2023-11-28T11:53:54.823652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:14:45.264482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357059+0000","last_change":"2023-11-28T11:53:53.351406+0000","last_active":"2023-11-28T11:53:53.357059+0000","last_peered":"2023-11-28T11:53:53.357059+0000","last_clean":"2023-11-28T11:53:53.357059+0000","last_became_active":"2023-11-28T11:53:53.350803+0000","last_became_peered":"2023-11-28T11:53:53.350803+0000","last_unstale":"2023-11-28T11:53:53.357059+0000","last_undegraded":"2023-11-28T11:53:53.357059+0000","last_fullsized":"2023-11-28T11:53:53.357059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:21.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.354302+0000","last_change":"2023-11-28T11:53:52.941710+0000","last_active":"2023-11-28T11:53:53.354302+0000","last_peered":"2023-11-28T11:53:53.354302+0000","last_clean":"2023-11-28T11:53:53.354302+0000","last_became_active":"2023-11-28T11:53:52.941111+0000","last_became_peered":"2023-11-28T11:53:52.941111+0000","last_unstale":"2023-11-28T11:53:53.354302+0000","last_undegraded":"2023-11-28T11:53:53.354302+0000","last_fullsized":"2023-11-28T11:53:53.354302+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:16:42.644755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.360999+0000","last_change":"2023-11-28T11:53:53.356586+0000","last_active":"2023-11-28T11:53:53.360999+0000","last_peered":"2023-11-28T11:53:53.360999+0000","last_clean":"2023-11-28T11:53:53.360999+0000","last_became_active":"2023-11-28T11:53:53.356292+0000","last_became_peered":"2023-11-28T11:53:53.356292+0000","last_unstale":"2023-11-28T11:53:53.360999+0000","last_undegraded":"2023-11-28T11:53:53.360999+0000","last_fullsized":"2023-11-28T11:53:53.360999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:07.501442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.346351+0000","last_change":"2023-11-28T11:53:52.945663+0000","last_active":"2023-11-28T11:53:53.346351+0000","last_peered":"2023-11-28T11:53:53.346351+0000","last_clean":"2023-11-28T11:53:53.346351+0000","last_became_active":"2023-11-28T11:53:52.945389+0000","last_became_peered":"2023-11-28T11:53:52.945389+0000","last_unstale":"2023-11-28T11:53:53.346351+0000","last_undegraded":"2023-11-28T11:53:53.346351+0000","last_fullsized":"2023-11-28T11:53:53.346351+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:06.499284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348469+0000","last_change":"2023-11-28T11:53:53.363683+0000","last_active":"2023-11-28T11:53:54.348469+0000","last_peered":"2023-11-28T11:53:54.348469+0000","last_clean":"2023-11-28T11:53:54.348469+0000","last_became_active":"2023-11-28T11:53:53.363336+0000","last_became_peered":"2023-11-28T11:53:53.363336+0000","last_unstale":"2023-11-28T11:53:54.348469+0000","last_undegraded":"2023-11-28T11:53:54.348469+0000","last_fullsized":"2023-11-28T11:53:54.348469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:24:40.402636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347720+0000","last_change":"2023-11-28T11:53:52.937233+0000","last_active":"2023-11-28T11:53:54.347720+0000","last_peered":"2023-11-28T11:53:54.347720+0000","last_clean":"2023-11-28T11:53:54.347720+0000","last_became_active":"2023-11-28T11:53:52.936952+0000","last_became_peered":"2023-11-28T11:53:52.936952+0000","last_unstale":"2023-11-28T11:53:54.347720+0000","last_undegraded":"2023-11-28T11:53:54.347720+0000","last_fullsized":"2023-11-28T11:53:54.347720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:20:46.490834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367320+0000","last_change":"2023-11-28T11:53:53.365718+0000","last_active":"2023-11-28T11:53:53.367320+0000","last_peered":"2023-11-28T11:53:53.367320+0000","last_clean":"2023-11-28T11:53:53.367320+0000","last_became_active":"2023-11-28T11:53:53.365501+0000","last_became_peered":"2023-11-28T11:53:53.365501+0000","last_unstale":"2023-11-28T11:53:53.367320+0000","last_undegraded":"2023-11-28T11:53:53.367320+0000","last_fullsized":"2023-11-28T11:53:53.367320+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:01.100398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347644+0000","last_change":"2023-11-28T11:53:52.940103+0000","last_active":"2023-11-28T11:53:54.347644+0000","last_peered":"2023-11-28T11:53:54.347644+0000","last_clean":"2023-11-28T11:53:54.347644+0000","last_became_active":"2023-11-28T11:53:52.939842+0000","last_became_peered":"2023-11-28T11:53:52.939842+0000","last_unstale":"2023-11-28T11:53:54.347644+0000","last_undegraded":"2023-11-28T11:53:54.347644+0000","last_fullsized":"2023-11-28T11:53:54.347644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:41.834173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.361370+0000","last_change":"2023-11-28T11:53:53.353001+0000","last_active":"2023-11-28T11:53:53.361370+0000","last_peered":"2023-11-28T11:53:53.361370+0000","last_clean":"2023-11-28T11:53:53.361370+0000","last_became_active":"2023-11-28T11:53:53.352771+0000","last_became_peered":"2023-11-28T11:53:53.352771+0000","last_unstale":"2023-11-28T11:53:53.361370+0000","last_undegraded":"2023-11-28T11:53:53.361370+0000","last_fullsized":"2023-11-28T11:53:53.361370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:55.715676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.353965+0000","last_change":"2023-11-28T11:53:52.943760+0000","last_active":"2023-11-28T11:53:53.353965+0000","last_peered":"2023-11-28T11:53:53.353965+0000","last_clean":"2023-11-28T11:53:53.353965+0000","last_became_active":"2023-11-28T11:53:52.943484+0000","last_became_peered":"2023-11-28T11:53:52.943484+0000","last_unstale":"2023-11-28T11:53:53.353965+0000","last_undegraded":"2023-11-28T11:53:53.353965+0000","last_fullsized":"2023-11-28T11:53:53.353965+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:36:39.945555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348626+0000","last_change":"2023-11-28T11:53:53.363345+0000","last_active":"2023-11-28T11:53:54.348626+0000","last_peered":"2023-11-28T11:53:54.348626+0000","last_clean":"2023-11-28T11:53:54.348626+0000","last_became_active":"2023-11-28T11:53:53.363053+0000","last_became_peered":"2023-11-28T11:53:53.363053+0000","last_unstale":"2023-11-28T11:53:54.348626+0000","last_undegraded":"2023-11-28T11:53:54.348626+0000","last_fullsized":"2023-11-28T11:53:54.348626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:56.344374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.365792+0000","last_change":"2023-11-28T11:53:52.949669+0000","last_active":"2023-11-28T11:53:53.365792+0000","last_peered":"2023-11-28T11:53:53.365792+0000","last_clean":"2023-11-28T11:53:53.365792+0000","last_became_active":"2023-11-28T11:53:52.949404+0000","last_became_peered":"2023-11-28T11:53:52.949404+0000","last_unstale":"2023-11-28T11:53:53.365792+0000","last_undegraded":"2023-11-28T11:53:53.365792+0000","last_fullsized":"2023-11-28T11:53:53.365792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:35.166918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.355760+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:53:53.355760+0000","last_peered":"2023-11-28T11:53:53.355760+0000","last_clean":"2023-11-28T11:53:53.355760+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:53:53.355760+0000","last_undegraded":"2023-11-28T11:53:53.355760+0000","last_fullsized":"2023-11-28T11:53:53.355760+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.357084+0000","last_change":"2023-11-28T11:53:53.351618+0000","last_active":"2023-11-28T11:53:53.357084+0000","last_peered":"2023-11-28T11:53:53.357084+0000","last_clean":"2023-11-28T11:53:53.357084+0000","last_became_active":"2023-11-28T11:53:53.351148+0000","last_became_peered":"2023-11-28T11:53:53.351148+0000","last_unstale":"2023-11-28T11:53:53.357084+0000","last_undegraded":"2023-11-28T11:53:53.357084+0000","last_fullsized":"2023-11-28T11:53:53.357084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:29:00.953015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347779+0000","last_change":"2023-11-28T11:53:52.939287+0000","last_active":"2023-11-28T11:53:54.347779+0000","last_peered":"2023-11-28T11:53:54.347779+0000","last_clean":"2023-11-28T11:53:54.347779+0000","last_became_active":"2023-11-28T11:53:52.938993+0000","last_became_peered":"2023-11-28T11:53:52.938993+0000","last_unstale":"2023-11-28T11:53:54.347779+0000","last_undegraded":"2023-11-28T11:53:54.347779+0000","last_fullsized":"2023-11-28T11:53:54.347779+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:27:42.936931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.345063+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:53:53.345063+0000","last_peered":"2023-11-28T11:53:53.345063+0000","last_clean":"2023-11-28T11:53:53.345063+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:53:53.345063+0000","last_undegraded":"2023-11-28T11:53:53.345063+0000","last_fullsized":"2023-11-28T11:53:53.345063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823334+0000","last_change":"2023-11-28T11:53:53.357600+0000","last_active":"2023-11-28T11:53:54.823334+0000","last_peered":"2023-11-28T11:53:54.823334+0000","last_clean":"2023-11-28T11:53:54.823334+0000","last_became_active":"2023-11-28T11:53:53.357045+0000","last_became_peered":"2023-11-28T11:53:53.357045+0000","last_unstale":"2023-11-28T11:53:54.823334+0000","last_undegraded":"2023-11-28T11:53:54.823334+0000","last_fullsized":"2023-11-28T11:53:54.823334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:30.873940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"59'2","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823410+0000","last_change":"2023-11-28T11:53:52.946969+0000","last_active":"2023-11-28T11:53:54.823410+0000","last_peered":"2023-11-28T11:53:54.823410+0000","last_clean":"2023-11-28T11:53:54.823410+0000","last_became_active":"2023-11-28T11:53:52.946640+0000","last_became_peered":"2023-11-28T11:53:52.946640+0000","last_unstale":"2023-11-28T11:53:54.823410+0000","last_undegraded":"2023-11-28T11:53:54.823410+0000","last_fullsized":"2023-11-28T11:53:54.823410+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:00.844589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":42,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.930508+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:53:52.930508+0000","last_peered":"2023-11-28T11:53:52.930508+0000","last_clean":"2023-11-28T11:53:52.930508+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:53:52.930508+0000","last_undegraded":"2023-11-28T11:53:52.930508+0000","last_fullsized":"2023-11-28T11:53:52.930508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":193,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.936228+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:53:52.936228+0000","last_peered":"2023-11-28T11:53:52.936228+0000","last_clean":"2023-11-28T11:53:52.936228+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:53:52.936228+0000","last_undegraded":"2023-11-28T11:53:52.936228+0000","last_fullsized":"2023-11-28T11:53:52.936228+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347470+0000","last_change":"2023-11-28T11:53:53.354642+0000","last_active":"2023-11-28T11:53:54.347470+0000","last_peered":"2023-11-28T11:53:54.347470+0000","last_clean":"2023-11-28T11:53:54.347470+0000","last_became_active":"2023-11-28T11:53:53.354347+0000","last_became_peered":"2023-11-28T11:53:53.354347+0000","last_unstale":"2023-11-28T11:53:54.347470+0000","last_undegraded":"2023-11-28T11:53:54.347470+0000","last_fullsized":"2023-11-28T11:53:54.347470+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:42.420815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347763+0000","last_change":"2023-11-28T11:53:52.944877+0000","last_active":"2023-11-28T11:53:54.347763+0000","last_peered":"2023-11-28T11:53:54.347763+0000","last_clean":"2023-11-28T11:53:54.347763+0000","last_became_active":"2023-11-28T11:53:52.944176+0000","last_became_peered":"2023-11-28T11:53:52.944176+0000","last_unstale":"2023-11-28T11:53:54.347763+0000","last_undegraded":"2023-11-28T11:53:54.347763+0000","last_fullsized":"2023-11-28T11:53:54.347763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:03:00.098240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.938643+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:53:52.938643+0000","last_peered":"2023-11-28T11:53:52.938643+0000","last_clean":"2023-11-28T11:53:52.938643+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:53:52.938643+0000","last_undegraded":"2023-11-28T11:53:52.938643+0000","last_fullsized":"2023-11-28T11:53:52.938643+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.361032+0000","last_change":"2023-11-28T11:53:53.354352+0000","last_active":"2023-11-28T11:53:53.361032+0000","last_peered":"2023-11-28T11:53:53.361032+0000","last_clean":"2023-11-28T11:53:53.361032+0000","last_became_active":"2023-11-28T11:53:53.354105+0000","last_became_peered":"2023-11-28T11:53:53.354105+0000","last_unstale":"2023-11-28T11:53:53.361032+0000","last_undegraded":"2023-11-28T11:53:53.361032+0000","last_fullsized":"2023-11-28T11:53:53.361032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:36.169075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.347564+0000","last_change":"2023-11-28T11:53:52.944777+0000","last_active":"2023-11-28T11:53:54.347564+0000","last_peered":"2023-11-28T11:53:54.347564+0000","last_clean":"2023-11-28T11:53:54.347564+0000","last_became_active":"2023-11-28T11:53:52.944431+0000","last_became_peered":"2023-11-28T11:53:52.944431+0000","last_unstale":"2023-11-28T11:53:54.347564+0000","last_undegraded":"2023-11-28T11:53:54.347564+0000","last_fullsized":"2023-11-28T11:53:54.347564+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:00.334786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":44,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.936124+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:53:52.936124+0000","last_peered":"2023-11-28T11:53:52.936124+0000","last_clean":"2023-11-28T11:53:52.936124+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:53:52.936124+0000","last_undegraded":"2023-11-28T11:53:52.936124+0000","last_fullsized":"2023-11-28T11:53:52.936124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.363412+0000","last_change":"2023-11-28T11:53:53.357550+0000","last_active":"2023-11-28T11:53:53.363412+0000","last_peered":"2023-11-28T11:53:53.363412+0000","last_clean":"2023-11-28T11:53:53.363412+0000","last_became_active":"2023-11-28T11:53:53.356972+0000","last_became_peered":"2023-11-28T11:53:53.356972+0000","last_unstale":"2023-11-28T11:53:53.363412+0000","last_undegraded":"2023-11-28T11:53:53.363412+0000","last_fullsized":"2023-11-28T11:53:53.363412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:06.472324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.823352+0000","last_change":"2023-11-28T11:53:52.948014+0000","last_active":"2023-11-28T11:53:54.823352+0000","last_peered":"2023-11-28T11:53:54.823352+0000","last_clean":"2023-11-28T11:53:54.823352+0000","last_became_active":"2023-11-28T11:53:52.947761+0000","last_became_peered":"2023-11-28T11:53:52.947761+0000","last_unstale":"2023-11-28T11:53:54.823352+0000","last_undegraded":"2023-11-28T11:53:54.823352+0000","last_fullsized":"2023-11-28T11:53:54.823352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:30:59.358348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.348383+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:53:53.348383+0000","last_peered":"2023-11-28T11:53:53.348383+0000","last_clean":"2023-11-28T11:53:53.348383+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:53:53.348383+0000","last_undegraded":"2023-11-28T11:53:53.348383+0000","last_fullsized":"2023-11-28T11:53:53.348383+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.371693+0000","last_change":"2023-11-28T11:53:53.368465+0000","last_active":"2023-11-28T11:53:53.371693+0000","last_peered":"2023-11-28T11:53:53.371693+0000","last_clean":"2023-11-28T11:53:53.371693+0000","last_became_active":"2023-11-28T11:53:53.367541+0000","last_became_peered":"2023-11-28T11:53:53.367541+0000","last_unstale":"2023-11-28T11:53:53.371693+0000","last_undegraded":"2023-11-28T11:53:53.371693+0000","last_fullsized":"2023-11-28T11:53:53.371693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:03:33.844830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"59'2","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.367522+0000","last_change":"2023-11-28T11:53:52.945782+0000","last_active":"2023-11-28T11:53:53.367522+0000","last_peered":"2023-11-28T11:53:53.367522+0000","last_clean":"2023-11-28T11:53:53.367522+0000","last_became_active":"2023-11-28T11:53:52.945501+0000","last_became_peered":"2023-11-28T11:53:52.945501+0000","last_unstale":"2023-11-28T11:53:53.367522+0000","last_undegraded":"2023-11-28T11:53:53.367522+0000","last_fullsized":"2023-11-28T11:53:53.367522+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:10.424141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.937142+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:53:52.937142+0000","last_peered":"2023-11-28T11:53:52.937142+0000","last_clean":"2023-11-28T11:53:52.937142+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:53:52.937142+0000","last_undegraded":"2023-11-28T11:53:52.937142+0000","last_fullsized":"2023-11-28T11:53:52.937142+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-28T11:53:53.366542+0000","last_change":"2023-11-28T11:53:53.365132+0000","last_active":"2023-11-28T11:53:53.366542+0000","last_peered":"2023-11-28T11:53:53.366542+0000","last_clean":"2023-11-28T11:53:53.366542+0000","last_became_active":"2023-11-28T11:53:53.364841+0000","last_became_peered":"2023-11-28T11:53:53.364841+0000","last_unstale":"2023-11-28T11:53:53.366542+0000","last_undegraded":"2023-11-28T11:53:53.366542+0000","last_fullsized":"2023-11-28T11:53:53.366542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:52.925665+0000","last_clean_scrub_stamp":"2023-11-28T11:53:52.925665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:12:18.360820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"59'1","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T11:53:54.348816+0000","last_change":"2023-11-28T11:53:52.944615+0000","last_active":"2023-11-28T11:53:54.348816+0000","last_peered":"2023-11-28T11:53:54.348816+0000","last_clean":"2023-11-28T11:53:54.348816+0000","last_became_active":"2023-11-28T11:53:52.944325+0000","last_became_peered":"2023-11-28T11:53:52.944325+0000","last_unstale":"2023-11-28T11:53:54.348816+0000","last_undegraded":"2023-11-28T11:53:54.348816+0000","last_fullsized":"2023-11-28T11:53:54.348816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:53:51.923508+0000","last_clean_scrub_stamp":"2023-11-28T11:53:51.923508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:05.470167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-28T11:53:52.937086+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:53:52.937086+0000","last_peered":"2023-11-28T11:53:52.937086+0000","last_clean":"2023-11-28T11:53:52.937086+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:53:52.937086+0000","last_undegraded":"2023-11-28T11:53:52.937086+0000","last_fullsized":"2023-11-28T11:53:52.937086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10700,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361140,"statfs":{"total":96636764160,"available":96625807360,"internally_reserved":0,"allocated":1314816,"data_stored":898513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.45700000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.62},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.66600000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.46300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.70199999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.49199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.497},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.74099999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61099999999999999,"5min":0.751,"15min":0.751},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.502,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72599999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:53:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71799999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.72099999999999997}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9276,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362564,"statfs":{"total":96636764160,"available":96627265536,"internally_reserved":0,"allocated":708608,"data_stored":307419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.61399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52900000000000003,"15min":0.53000000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.629},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70699999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59099999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68000000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70999999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":3.7810000000000001,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.73699999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65100000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.46300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.44500000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72399999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51800000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10456,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361384,"statfs":{"total":96636764160,"available":96626057216,"internally_reserved":0,"allocated":1327104,"data_stored":900049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.32600000000000001,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.42399999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.35899999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35299999999999998,"15min":0.35199999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.499,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.30499999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38300000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.44,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80200000000000005,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76200000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.49299999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.625},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83499999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48299999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13568,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358272,"statfs":{"total":96636764160,"available":96622870528,"internally_reserved":0,"allocated":712704,"data_stored":313052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.316,"15min":0.31},"min":{"1min":0.22,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.436},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.34899999999999998},"min":{"1min":0.253,"5min":0.251,"15min":0.251},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.29299999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.30099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.44600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.45600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.49099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54500000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.439,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.505}]},{"osd":7,"last update":"Tue Nov 28 11:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.44500000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.502,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51700000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9336,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362504,"statfs":{"total":96636764160,"available":96627204096,"internally_reserved":0,"allocated":704512,"data_stored":307385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.47999999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.38,"5min":0.66800000000000004,"15min":0.71599999999999997},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.57299999999999995,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.434}]},{"osd":1,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.30599999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48999999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.68700000000000006,"15min":0.73599999999999999},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34200000000000003,"15min":0.33600000000000002},"min":{"1min":0.25900000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.502,"5min":0.622,"15min":0.622},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.67700000000000005,"15min":0.71899999999999997},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57599999999999996,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42899999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.91700000000000004,"15min":0.96399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":8.359,"15min":8.359},"last":0.749}]},{"osd":5,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.86199999999999999,"15min":0.89800000000000002},"min":{"1min":0.44800000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82999999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.90400000000000003,"15min":0.94599999999999995},"min":{"1min":0.438,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84899999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.93500000000000005,"15min":0.97999999999999998},"min":{"1min":0.40500000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.75800000000000001,"15min":0.78000000000000003},"min":{"1min":0.45300000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82199999999999995,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.79100000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9720,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362120,"statfs":{"total":96636764160,"available":96626810880,"internally_reserved":0,"allocated":704512,"data_stored":307385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.25,"5min":0.224,"15min":0.224},"max":{"1min":0.46400000000000002,"5min":1.762,"15min":1.762},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60599999999999998,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.317}]},{"osd":1,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.96699999999999997,"15min":1.0669999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55500000000000005,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.29199999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61899999999999999,"5min":1.857,"15min":1.857},"last":0.33700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.39200000000000002,"15min":0.39000000000000001},"min":{"1min":0.30099999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.56200000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.70299999999999996,"15min":0.749},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.60699999999999998,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.93200000000000005,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.82499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88700000000000001,"5min":1.498,"15min":1.498},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.66300000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86699999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.84599999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.98999999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89100000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.55000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9540,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362300,"statfs":{"total":96636764160,"available":96626995200,"internally_reserved":0,"allocated":716800,"data_stored":308933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.56499999999999995,"15min":0.58099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67300000000000004,"5min":2.137,"15min":2.137},"last":0.51800000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.501,"15min":0.5},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.499,"15min":0.501},"min":{"1min":0.378,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59399999999999997,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.46800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.44700000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33600000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69699999999999995,"5min":0.745,"15min":0.745},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.629,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.35799999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.30499999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65500000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.433,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76100000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.497}]},{"osd":6,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.60699999999999998,"15min":0.623},"min":{"1min":0.36499999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71799999999999997,"5min":2.286,"15min":2.286},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70799999999999996,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:53:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.64800000000000002,"15min":0.66400000000000003},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67300000000000004,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73499999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53700000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9400,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362440,"statfs":{"total":96636764160,"available":96627138560,"internally_reserved":0,"allocated":704512,"data_stored":306421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77400000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.40100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.51800000000000002},"min":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79200000000000004,"5min":1.679,"15min":1.679},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.504}]},{"osd":4,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.373,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79400000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.55400000000000005},"min":{"1min":0.39000000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.76300000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.623},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.371,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:53:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76000000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:53:57.797 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:53:57.798 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:53:57.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.957+0000 7f53ff8ce700 1 Processor -- start 2023-11-28T11:53:57.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.957+0000 7f53ff8ce700 1 -- start start 2023-11-28T11:53:57.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53ff8ce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f8140950 0x7f53f80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53ff8ce700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f53f80b2e70 0x7f53f806f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53ff8ce700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f813ff00 con 0x7f53f80cd8c0 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53ff8ce700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f81404c0 con 0x7f53f80b2e70 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53ff8ce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8140dc0 con 0x7f53f8140950 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fd66a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f53f80cd8c0 0x7f53f80cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55212/0 (socket says 172.21.15.144:55212) 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fd66a700 1 -- 172.21.15.144:0/1433048295 learned_addr learned my addr 172.21.15.144:0/1433048295 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fce69700 1 --2- 172.21.15.144:0/1433048295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f8140950 0x7f53f80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fde6b700 1 --2- 172.21.15.144:0/1433048295 >> v2:172.21.15.177:3300/0 conn(0x7f53f80b2e70 0x7f53f806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53effff700 1 -- 172.21.15.144:0/1433048295 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219459854 0 0) 0x7f53f813ff00 con 0x7f53f80cd8c0 2023-11-28T11:53:57.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53effff700 1 -- 172.21.15.144:0/1433048295 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e40035b0 con 0x7f53f80cd8c0 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fce69700 1 -- 172.21.15.144:0/1433048295 >> v2:172.21.15.177:3300/0 conn(0x7f53f80b2e70 msgr2=0x7f53f806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fce69700 1 --2- 172.21.15.144:0/1433048295 >> v2:172.21.15.177:3300/0 conn(0x7f53f80b2e70 0x7f53f806f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fce69700 1 -- 172.21.15.144:0/1433048295 >> v1:172.21.15.144:6789/0 conn(0x7f53f80cd8c0 legacy=0x7f53f80cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fce69700 1 -- 172.21.15.144:0/1433048295 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f815aa80 con 0x7f53f8140950 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.958+0000 7f53fce69700 1 --2- 172.21.15.144:0/1433048295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f8140950 0x7f53f80b2930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009840 tx=0x7f53e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1749765952cc5be9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53effff700 1 -- 172.21.15.144:0/1433048295 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8011830 con 0x7f53f8140950 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53effff700 1 -- 172.21.15.144:0/1433048295 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e8011990 con 0x7f53f8140950 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53effff700 1 -- 172.21.15.144:0/1433048295 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8011d20 con 0x7f53f8140950 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 -- 172.21.15.144:0/1433048295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f8140950 msgr2=0x7f53f80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/1433048295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f8140950 0x7f53f80b2930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009840 tx=0x7f53e8009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 -- 172.21.15.144:0/1433048295 shutdown_connections 2023-11-28T11:53:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/1433048295 >> v2:172.21.15.177:3300/0 conn(0x7f53f80b2e70 0x7f53f806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/1433048295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f8140950 0x7f53f80b2930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 -- 172.21.15.144:0/1433048295 >> 172.21.15.144:0/1433048295 conn(0x7f53f806aed0 msgr2=0x7f53f806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 -- 172.21.15.144:0/1433048295 shutdown_connections 2023-11-28T11:53:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.959+0000 7f53ff8ce700 1 -- 172.21.15.144:0/1433048295 wait complete. 2023-11-28T11:53:57.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.961+0000 7f53ff8ce700 1 Processor -- start 2023-11-28T11:53:57.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.961+0000 7f53ff8ce700 1 -- start start 2023-11-28T11:53:57.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53f80b2e70 0x7f53f8157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f80cd8c0 0x7f53f813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 0x7f53f813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8143f80 con 0x7f53f8140950 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f806cda0 con 0x7f53f80b2e70 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f806c190 con 0x7f53f80cd8c0 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 0x7f53f813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 0x7f53f813d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38510/0 (socket says 172.21.15.144:38510) 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 -- 172.21.15.144:0/2380410049 learned_addr learned my addr 172.21.15.144:0/2380410049 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 -- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f80cd8c0 msgr2=0x7f53f813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fce69700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f80cd8c0 0x7f53f813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fd66a700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53f80b2e70 0x7f53f8157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f80cd8c0 0x7f53f813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 -- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53f80b2e70 msgr2=0x7f53f8157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53f80b2e70 0x7f53f8157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 -- 172.21.15.144:0/2380410049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80094a0 con 0x7f53f8140950 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fce69700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f80cd8c0 0x7f53f813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fd66a700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53f80b2e70 0x7f53f8157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53fde6b700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 0x7f53f813d230 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f53f000ef90 tx=0x7f53f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0009ca0 con 0x7f53f8140950 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f000f040 con 0x7f53f8140950 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f80b35f0 con 0x7f53f8140950 2023-11-28T11:53:57.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.962+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0014480 con 0x7f53f8140950 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.963+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f80b3310 con 0x7f53f8140950 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.964+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f8140150 con 0x7f53f8140950 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.964+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f53f00145e0 con 0x7f53f8140950 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.964+0000 7f53edffb700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53dc0682f0 0x7f53dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.964+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53f0086810 con 0x7f53f8140950 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.964+0000 7f53fd66a700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53dc0682f0 0x7f53dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:57.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.965+0000 7f53fd66a700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53dc0682f0 0x7f53dc06a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53f40010e0 tx=0x7f53f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:57.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:57.968+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f53f0053070 con 0x7f53f8140950 2023-11-28T11:53:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.228+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f53f8068360 con 0x7f53f8140950 2023-11-28T11:53:58.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.232+0000 7f53edffb700 1 -- 172.21.15.144:0/2380410049 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f53f0056690 con 0x7f53f8140950 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53dc0682f0 msgr2=0x7f53dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53dc0682f0 0x7f53dc06a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53f40010e0 tx=0x7f53f4009040 comp rx=0 tx=0).stop 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 msgr2=0x7f53f813d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 0x7f53f813d230 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f53f000ef90 tx=0x7f53f000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 shutdown_connections 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53f8140950 0x7f53f813d230 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f53dc0682f0 0x7f53dc06a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53f80b2e70 0x7f53f8157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 --2- 172.21.15.144:0/2380410049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53f80cd8c0 0x7f53f813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 >> 172.21.15.144:0/2380410049 conn(0x7f53f806aed0 msgr2=0x7f53f80cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:58.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 shutdown_connections 2023-11-28T11:53:58.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.234+0000 7f53ff8ce700 1 -- 172.21.15.144:0/2380410049 wait complete. 2023-11-28T11:53:58.238 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.0 2023-11-28T11:53:58.248 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:53:58.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.367+0000 7f958d27c700 1 Processor -- start 2023-11-28T11:53:58.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.367+0000 7f958d27c700 1 -- start start 2023-11-28T11:53:58.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f958d27c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f958815c9b0 0x7f95880b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:58.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f958d27c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f95880cd7c0 0x7f95880cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:58.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f958d27c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880ce160 con 0x7f9588161340 2023-11-28T11:53:58.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f958d27c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880b5800 con 0x7f95880cd7c0 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f958d27c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880d1950 con 0x7f958815c9b0 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9586ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f958815c9b0 0x7f95880b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f95867fc700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f9588161340 0x7f95880d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:55716/0 (socket says 172.21.15.144:55716) 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f95867fc700 1 -- 172.21.15.144:0/2559095010 learned_addr learned my addr 172.21.15.144:0/2559095010 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f95877fe700 1 --2- 172.21.15.144:0/2559095010 >> v2:172.21.15.177:3300/0 conn(0x7f95880cd7c0 0x7f95880cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9585ffb700 1 -- 172.21.15.144:0/2559095010 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776097634 0 0) 0x7f95880ce160 con 0x7f9588161340 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9585ffb700 1 -- 172.21.15.144:0/2559095010 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95700035b0 con 0x7f9588161340 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9586ffd700 1 -- 172.21.15.144:0/2559095010 >> v2:172.21.15.177:3300/0 conn(0x7f95880cd7c0 msgr2=0x7f95880cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9586ffd700 1 --2- 172.21.15.144:0/2559095010 >> v2:172.21.15.177:3300/0 conn(0x7f95880cd7c0 0x7f95880cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9586ffd700 1 -- 172.21.15.144:0/2559095010 >> v1:172.21.15.177:6789/0 conn(0x7f9588161340 legacy=0x7f95880d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f9586ffd700 1 -- 172.21.15.144:0/2559095010 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880d24f0 con 0x7f958815c9b0 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.368+0000 7f95877fe700 1 --2- 172.21.15.144:0/2559095010 >> v2:172.21.15.177:3300/0 conn(0x7f95880cd7c0 0x7f95880cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:53:58.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f9586ffd700 1 --2- 172.21.15.144:0/2559095010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f958815c9b0 0x7f95880b4bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f957c009840 tx=0x7f957c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b42d45613fc0f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f9585ffb700 1 -- 172.21.15.144:0/2559095010 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c011830 con 0x7f958815c9b0 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f9585ffb700 1 -- 172.21.15.144:0/2559095010 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c011990 con 0x7f958815c9b0 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f9585ffb700 1 -- 172.21.15.144:0/2559095010 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c011d20 con 0x7f958815c9b0 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 -- 172.21.15.144:0/2559095010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f958815c9b0 msgr2=0x7f95880b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 --2- 172.21.15.144:0/2559095010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f958815c9b0 0x7f95880b4bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f957c009840 tx=0x7f957c009b50 comp rx=0 tx=0).stop 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 -- 172.21.15.144:0/2559095010 shutdown_connections 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 --2- 172.21.15.144:0/2559095010 >> v2:172.21.15.177:3300/0 conn(0x7f95880cd7c0 0x7f95880cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 --2- 172.21.15.144:0/2559095010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f958815c9b0 0x7f95880b4bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 -- 172.21.15.144:0/2559095010 >> 172.21.15.144:0/2559095010 conn(0x7f9588072600 msgr2=0x7f95880cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 -- 172.21.15.144:0/2559095010 shutdown_connections 2023-11-28T11:53:58.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.369+0000 7f958d27c700 1 -- 172.21.15.144:0/2559095010 wait complete. 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.371+0000 7f958d27c700 1 Processor -- start 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.371+0000 7f958d27c700 1 -- start start 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95880cd7c0 0x7f95880c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958815c9b0 0x7f95880cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 0x7f9588157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880b8bc0 con 0x7f9588161340 2023-11-28T11:53:58.376 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880b7d10 con 0x7f958815c9b0 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880b7fb0 con 0x7f95880cd7c0 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 0x7f9588157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 0x7f9588157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38524/0 (socket says 172.21.15.144:38524) 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 -- 172.21.15.144:0/2629962991 learned_addr learned my addr 172.21.15.144:0/2629962991 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95867fc700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958815c9b0 0x7f95880cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f9586ffd700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95880cd7c0 0x7f95880c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 -- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95880cd7c0 msgr2=0x7f95880c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95880cd7c0 0x7f95880c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 -- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958815c9b0 msgr2=0x7f95880cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958815c9b0 0x7f95880cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.377 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 -- 172.21.15.144:0/2629962991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580009a70 con 0x7f9588161340 2023-11-28T11:53:58.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f95877fe700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 0x7f9588157350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f958000efc0 tx=0x7f958000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580009cd0 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000f040 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c0094a0 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000cd60 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.372+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95880d1300 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.373+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95880b5520 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.374+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9580014480 con 0x7f9588161340 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.374+0000 7f956f7fe700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f95740682f0 0x7f957406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:58.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.374+0000 7f9586ffd700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f95740682f0 0x7f957406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:58.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.374+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9580086500 con 0x7f9588161340 2023-11-28T11:53:58.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.375+0000 7f9586ffd700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f95740682f0 0x7f957406a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f957c000c00 tx=0x7f957c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:58.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.378+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9580052d00 con 0x7f9588161340 2023-11-28T11:53:58.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.605+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95880d15e0 con 0x7f9588161340 2023-11-28T11:53:58.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.606+0000 7f956f7fe700 1 -- 172.21.15.144:0/2629962991 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f9580056320 con 0x7f9588161340 2023-11-28T11:53:58.608 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:53:58.608 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:53:56.350260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f95740682f0 msgr2=0x7f957406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f95740682f0 0x7f957406a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f957c000c00 tx=0x7f957c01c040 comp rx=0 tx=0).stop 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 msgr2=0x7f9588157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 0x7f9588157350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f958000efc0 tx=0x7f958000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 shutdown_connections 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9588161340 0x7f9588157350 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f95740682f0 0x7f957406a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95880cd7c0 0x7f95880c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 --2- 172.21.15.144:0/2629962991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958815c9b0 0x7f95880cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 >> 172.21.15.144:0/2629962991 conn(0x7f9588072600 msgr2=0x7f958815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 shutdown_connections 2023-11-28T11:53:58.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:58.608+0000 7f958d27c700 1 -- 172.21.15.144:0/2629962991 wait complete. 2023-11-28T11:53:58.611 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 31 2023-11-28T11:53:59.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:53:53.337934+0000', 'modified': '2023-11-28T11:53:56.350260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5988}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5988': {'gid': 5988, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6837/3077123372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 3077123372}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 3077123372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:53:59.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:53:59.024 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T11:53:59.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 Processor -- start 2023-11-28T11:53:59.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 -- start start 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6dbc06fb30 0x7f6dbc158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dbc0cdda0 0x7f6dbc0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc06bf10 con 0x7f6dbc15d1f0 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc06ff00 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.144+0000 7f6dc19f7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b0df0 con 0x7f6dbc06fb30 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dbc0cdda0 0x7f6dbc0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dbc0cdda0 0x7f6dbc0ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38532/0 (socket says 172.21.15.144:38532) 2023-11-28T11:53:59.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/231483871 learned_addr learned my addr 172.21.15.144:0/231483871 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dba7fc700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6dbc15d1f0 0x7f6dbc0b06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55242/0 (socket says 172.21.15.144:55242) 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/231483871 >> v2:172.21.15.177:3300/0 conn(0x7f6dbc06fb30 0x7f6dbc158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/231483871 >> v2:172.21.15.177:3300/0 conn(0x7f6dbc06fb30 msgr2=0x7f6dbc158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/231483871 >> v2:172.21.15.177:3300/0 conn(0x7f6dbc06fb30 0x7f6dbc158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/231483871 >> v1:172.21.15.144:6789/0 conn(0x7f6dbc15d1f0 legacy=0x7f6dbc0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/231483871 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0ce6f0 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6db9ffb700 1 -- 172.21.15.144:0/231483871 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095380052 0 0) 0x7f6dbc06bf10 con 0x7f6dbc15d1f0 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/231483871 >> v2:172.21.15.177:3300/0 conn(0x7f6dbc06fb30 0x7f6dbc158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/231483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc0ce1b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc06fa30 tx=0x7f6db400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641c52846e6b1516 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6db9ffb700 1 -- 172.21.15.144:0/231483871 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db400e070 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6db9ffb700 1 -- 172.21.15.144:0/231483871 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db400b980 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.145+0000 7f6db9ffb700 1 -- 172.21.15.144:0/231483871 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db400bd30 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 -- 172.21.15.144:0/231483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 msgr2=0x7f6dbc0ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/231483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc0ce1b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc06fa30 tx=0x7f6db400b330 comp rx=0 tx=0).stop 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 -- 172.21.15.144:0/231483871 shutdown_connections 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/231483871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc0ce1b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/231483871 >> v2:172.21.15.177:3300/0 conn(0x7f6dbc06fb30 0x7f6dbc158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 -- 172.21.15.144:0/231483871 >> 172.21.15.144:0/231483871 conn(0x7f6dbc0cb670 msgr2=0x7f6dbc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 -- 172.21.15.144:0/231483871 shutdown_connections 2023-11-28T11:53:59.150 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.146+0000 7f6dc19f7700 1 -- 172.21.15.144:0/231483871 wait complete. 2023-11-28T11:53:59.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.147+0000 7f6dc19f7700 1 Processor -- start 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 -- start start 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc06fb30 0x7f6dbc0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc15d1f0 0x7f6dbc070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc140df0 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b3a90 con 0x7f6dbc15d1f0 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dc19f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b3d70 con 0x7f6dbc06fb30 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.148+0000 7f6dba7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38536/0 (socket says 172.21.15.144:38536) 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 -- 172.21.15.144:0/2998481116 learned_addr learned my addr 172.21.15.144:0/2998481116 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc15d1f0 0x7f6dbc070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc06fb30 0x7f6dbc0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 -- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc06fb30 msgr2=0x7f6dbc0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc06fb30 0x7f6dbc0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 -- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc15d1f0 msgr2=0x7f6dbc070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc15d1f0 0x7f6dbc070580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 -- 172.21.15.144:0/2998481116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4009b30 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc070040 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c550 tx=0x7f6dac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6da3fff700 1 -- 172.21.15.144:0/2998481116 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac004c80 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6da3fff700 1 -- 172.21.15.144:0/2998481116 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac004e00 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc0cb3b0 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6da3fff700 1 -- 172.21.15.144:0/2998481116 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac007310 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.149+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc0cb0d0 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.151+0000 7f6da3fff700 1 -- 172.21.15.144:0/2998481116 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6dac01a020 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.151+0000 7f6da3fff700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6da80682f0 0x7f6da806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.151+0000 7f6da3fff700 1 -- 172.21.15.144:0/2998481116 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6dac0858b0 con 0x7f6dbc0cdda0 2023-11-28T11:53:59.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.151+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6da80682f0 0x7f6da806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.151+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6da80682f0 0x7f6da806a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6db0006fd0 tx=0x7f6db0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.153+0000 7f6dcdb80b80 1 Processor -- start 2023-11-28T11:53:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.153+0000 7f6dcdb80b80 1 -- start start 2023-11-28T11:53:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x55890244abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x558902441a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.158 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902441f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890230abc0 con 0x55890244b120 2023-11-28T11:53:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902379350 con 0x55890241d210 2023-11-28T11:53:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x558902372880 con 0x55890244a810 2023-11-28T11:53:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902441f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902441f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38546/0 (socket says 172.21.15.144:38546) 2023-11-28T11:53:59.159 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2635403864 learned_addr learned my addr 172.21.15.144:0/2635403864 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x558902441a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x55890244abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 msgr2=0x55890244abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x55890244abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 msgr2=0x558902441a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.160 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x558902441a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2635403864 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589021f45e0 con 0x55890244b120 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.154+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902441f50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc067360 tx=0x7f6db4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f79908fae1e9019 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6da17fa700 1 -- 172.21.15.144:0/2635403864 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4003a20 con 0x55890244b120 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6da17fa700 1 -- 172.21.15.144:0/2635403864 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db4003b80 con 0x55890244b120 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6da17fa700 1 -- 172.21.15.144:0/2635403864 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4004040 con 0x55890244b120 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2635403864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 msgr2=0x558902441f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902441f50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc067360 tx=0x7f6db4010000 comp rx=0 tx=0).stop 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2635403864 shutdown_connections 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902441f50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x55890244abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2635403864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x558902441a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2635403864 >> 172.21.15.144:0/2635403864 conn(0x55890241dd60 msgr2=0x55890241e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2635403864 shutdown_connections 2023-11-28T11:53:59.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.155+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2635403864 wait complete. 2023-11-28T11:53:59.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.156+0000 7f6dcdb80b80 1 -- start start 2023-11-28T11:53:59.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.156+0000 7f6dcdb80b80 1 Processor -- start 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x5589024658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x558902438e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x55890244d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589021f68d0 con 0x55890244b120 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890244e800 con 0x55890241d210 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x558902465e20 con 0x55890244a810 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dcdb80b80 10 client.0 _ll_register_callbacks cb 0x558902218240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:59.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x55890244d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x55890244d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38552/0 (socket says 172.21.15.144:38552) 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/3135534408 learned_addr learned my addr 172.21.15.144:0/3135534408 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x5589024658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x558902438e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 msgr2=0x558902438e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x558902438e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 msgr2=0x5589024658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x5589024658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/3135534408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc0ce6f0 con 0x55890244b120 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x558902438e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:59.167 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x55890244d570 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b370 tx=0x7f6db4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d19c9125b9246d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.157+0000 7f6d9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/3135534408 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4004450 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/3135534408 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6db402fc50 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558902340140 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/3135534408 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db402fe10 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 10 client.6129 fetch_fsmap learned FSMap version 31 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x5589021a0d90 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/3135534408 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558902340140 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/3135534408 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6db4023030 con 0x55890244b120 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 10 client.6129 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 10 client.6129 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:53:59.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 20 client.6129 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5589024637d0 0x558902463ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 4 client.6129 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589021f6f10 con 0x5589024637d0 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5589024637d0 0x558902463ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6d9a7fc700 10 client.6129 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.158+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5589024637d0 0x558902463ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74aceeda38389a96 server_cookie=9b11a65ddb3e9bc7 in_seq=0 out_seq=0 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/3135534408 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5589021f6f10 con 0x5589024637d0 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6d9a7fc700 10 client.6129 handle_command_reply: tid=0 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 client.6129 shutdown 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 msgr2=0x55890244d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x55890244d570 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b370 tx=0x7f6db4004b40 comp rx=0 tx=0).stop 2023-11-28T11:53:59.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 shutdown_connections 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5589024637d0 0x558902463ba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x55890244d570 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890241d210 0x5589024658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/3135534408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x558902438e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6d9a7fc700 0 client.6129 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 >> 172.21.15.144:0/3135534408 conn(0x55890241dd60 msgr2=0x55890241f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 shutdown_connections 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/3135534408 wait complete. 2023-11-28T11:53:59.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.159+0000 7f6dcdb80b80 20 client.6129 trim_cache size 0 max 16384 2023-11-28T11:53:59.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.177+0000 7f6dcdb80b80 1 Processor -- start 2023-11-28T11:53:59.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.177+0000 7f6dcdb80b80 1 -- start start 2023-11-28T11:53:59.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890241d210 0x558902489b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244a810 0x55890241a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902444cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890244ce60 con 0x55890244b120 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589024200a0 con 0x55890244a810 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55890241b990 con 0x55890241d210 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902444cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902444cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38560/0 (socket says 172.21.15.144:38560) 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2789139377 learned_addr learned my addr 172.21.15.144:0/2789139377 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244a810 0x55890241a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890241d210 0x558902489b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890241d210 msgr2=0x558902489b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890241d210 0x558902489b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244a810 msgr2=0x55890241a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244a810 0x55890241a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 -- 172.21.15.144:0/2789139377 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558902339140 con 0x55890244b120 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902444cf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6db400d010 tx=0x7f6db4027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be2e787cc12fa93 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6da1ffb700 1 -- 172.21.15.144:0/2789139377 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4004450 con 0x55890244b120 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6da1ffb700 1 -- 172.21.15.144:0/2789139377 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db40046a0 con 0x55890244b120 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.178+0000 7f6da1ffb700 1 -- 172.21.15.144:0/2789139377 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db402e640 con 0x55890244b120 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2789139377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 msgr2=0x558902444cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902444cf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6db400d010 tx=0x7f6db4027040 comp rx=0 tx=0).stop 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2789139377 shutdown_connections 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890244b120 0x558902444cf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890241d210 0x558902489b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/2789139377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244a810 0x55890241a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2789139377 >> 172.21.15.144:0/2789139377 conn(0x55890241dd60 msgr2=0x558902420920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2789139377 shutdown_connections 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.179+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/2789139377 wait complete. 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.180+0000 7f6dcdb80b80 1 -- start start 2023-11-28T11:53:59.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.180+0000 7f6dcdb80b80 1 Processor -- start 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 0x5589024ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x5589024791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244b120 0x5589024d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902479a60 con 0x55890241d210 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902463ea0 con 0x55890244b120 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55890246acd0 con 0x55890244a810 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dcdb80b80 10 client.0 _ll_register_callbacks cb 0x558902218240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:53:59.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 0x5589024ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 0x5589024ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38562/0 (socket says 172.21.15.144:38562) 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 -- 172.21.15.144:0/581666173 learned_addr learned my addr 172.21.15.144:0/581666173 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x5589024791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbb7fe700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244b120 0x5589024d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 -- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 msgr2=0x5589024791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x5589024791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 -- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244b120 msgr2=0x5589024d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244b120 0x5589024d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dbaffd700 1 -- 172.21.15.144:0/581666173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589021a0d90 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.181+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x5589024791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dbaffd700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 0x5589024ce720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x558902462fd0 tx=0x7f6db00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ce27053790f80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6d9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/581666173 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000eca0 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/581666173 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6db000ee00 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558902330160 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/581666173 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db00269b0 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 10 client.6135 fetch_fsmap learned FSMap version 31 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x5589021c7200 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/581666173 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558902330160 con 0x55890241d210 2023-11-28T11:53:59.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/581666173 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6db0026b10 con 0x55890241d210 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 10 client.6135 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 10 client.6135 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 20 client.6135 populate_metadata read hostname 'smithi144' 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x558902444cf0 0x5589024450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 4 client.6135 mds_command: new command op to 5988 tid=0[{"prefix": "status"}] 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55890231a170 con 0x558902444cf0 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.182+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x558902444cf0 0x5589024450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dba7fc700 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x558902444cf0 0x5589024450c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbec89ce5ce855bd server_cookie=2e2d4b7886b72cff in_seq=0 out_seq=0 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6d9a7fc700 10 client.6135 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6d9a7fc700 1 -- 172.21.15.144:0/581666173 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55890231a170 con 0x558902444cf0 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6d9a7fc700 10 client.6135 handle_command_reply: tid=0 2023-11-28T11:53:59.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 client.6135 shutdown 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 msgr2=0x5589024ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 0x5589024ce720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x558902462fd0 tx=0x7f6db00047c0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 shutdown_connections 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x558902444cf0 0x5589024450c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890241d210 0x5589024ce720 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55890244b120 0x5589024d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6d9a7fc700 0 client.6135 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 --2- 172.21.15.144:0/581666173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55890244a810 0x5589024791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 >> 172.21.15.144:0/581666173 conn(0x55890241dd60 msgr2=0x55890244c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 shutdown_connections 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 1 -- 172.21.15.144:0/581666173 wait complete. 2023-11-28T11:53:59.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.183+0000 7f6dcdb80b80 20 client.6135 trim_cache size 0 max 16384 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "cluster_fsid": "af726c13-c3f1-4279-9fca-12c6e4621b18", 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "whoami": 0, 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 5988, 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "want_state": "up:active", 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "state": "up:active", 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs", 2023-11-28T11:53:59.194 INFO:teuthology.orchestra.run.smithi144.stdout: "rank_uptime": 5.8448381559999998, 2023-11-28T11:53:59.195 INFO:teuthology.orchestra.run.smithi144.stdout: "mdsmap_epoch": 31, 2023-11-28T11:53:59.195 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch": 60, 2023-11-28T11:53:59.195 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_epoch_barrier": 60, 2023-11-28T11:53:59.195 INFO:teuthology.orchestra.run.smithi144.stdout: "uptime": 11.198063844 2023-11-28T11:53:59.195 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6da80682f0 msgr2=0x7f6da806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6da80682f0 0x7f6da806a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6db0006fd0 tx=0x7f6db0008040 comp rx=0 tx=0).stop 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 msgr2=0x7f6dbc070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc070040 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c550 tx=0x7f6dac00c910 comp rx=0 tx=0).stop 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 shutdown_connections 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f6da80682f0 0x7f6da806a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6dbc15d1f0 0x7f6dbc070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6dbc06fb30 0x7f6dbc0b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 --2- 172.21.15.144:0/2998481116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dbc0cdda0 0x7f6dbc070040 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 >> 172.21.15.144:0/2998481116 conn(0x7f6dbc0cb670 msgr2=0x7f6dbc0ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 shutdown_connections 2023-11-28T11:53:59.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.184+0000 7f6dc19f7700 1 -- 172.21.15.144:0/2998481116 wait complete. 2023-11-28T11:53:59.198 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:53:59.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.316+0000 7fe19d473700 1 Processor -- start 2023-11-28T11:53:59.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.316+0000 7fe19d473700 1 -- start start 2023-11-28T11:53:59.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe19d473700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe19d473700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1980d2320 0x7fe1980cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe19d473700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980d1950 con 0x7fe19815c9b0 2023-11-28T11:53:59.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe19d473700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1980b5800 con 0x7fe1980d2320 2023-11-28T11:53:59.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe19d473700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1980b5300 con 0x7fe198161340 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1977fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1980d2320 0x7fe1980cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe196ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe19815c9b0 0x7fe1980b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55282/0 (socket says 172.21.15.144:55282) 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe196ffd700 1 -- 172.21.15.144:0/3266745865 learned_addr learned my addr 172.21.15.144:0/3266745865 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1967fc700 1 --2- 172.21.15.144:0/3266745865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1977fe700 1 -- 172.21.15.144:0/3266745865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe198161340 msgr2=0x7fe1980d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1977fe700 1 --2- 172.21.15.144:0/3266745865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1977fe700 1 -- 172.21.15.144:0/3266745865 >> v1:172.21.15.144:6789/0 conn(0x7fe19815c9b0 legacy=0x7fe1980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1977fe700 1 -- 172.21.15.144:0/3266745865 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980b5520 con 0x7fe1980d2320 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.317+0000 7fe1977fe700 1 --2- 172.21.15.144:0/3266745865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1980d2320 0x7fe1980cd7c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe190009dc0 tx=0x7fe19000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78140b6c77a59647 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe195ffb700 1 -- 172.21.15.144:0/3266745865 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000e070 con 0x7fe1980d2320 2023-11-28T11:53:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe195ffb700 1 -- 172.21.15.144:0/3266745865 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19000bb90 con 0x7fe1980d2320 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe195ffb700 1 -- 172.21.15.144:0/3266745865 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190012430 con 0x7fe1980d2320 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 -- 172.21.15.144:0/3266745865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1980d2320 msgr2=0x7fe1980cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 --2- 172.21.15.144:0/3266745865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1980d2320 0x7fe1980cd7c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe190009dc0 tx=0x7fe19000b330 comp rx=0 tx=0).stop 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 -- 172.21.15.144:0/3266745865 shutdown_connections 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 --2- 172.21.15.144:0/3266745865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1980d2320 0x7fe1980cd7c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 --2- 172.21.15.144:0/3266745865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe198161340 0x7fe1980d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 -- 172.21.15.144:0/3266745865 >> 172.21.15.144:0/3266745865 conn(0x7fe198072600 msgr2=0x7fe1980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 -- 172.21.15.144:0/3266745865 shutdown_connections 2023-11-28T11:53:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.318+0000 7fe19d473700 1 -- 172.21.15.144:0/3266745865 wait complete. 2023-11-28T11:53:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 Processor -- start 2023-11-28T11:53:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 -- start start 2023-11-28T11:53:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1980d2320 0x7fe1981522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 0x7fe1981592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe198161340 0x7fe1981597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe198160040 con 0x7fe19815c9b0 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe198151e20 con 0x7fe198161340 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe19d473700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19815fd60 con 0x7fe1980d2320 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe1967fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 0x7fe1981592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe1967fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 0x7fe1981592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38578/0 (socket says 172.21.15.144:38578) 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.320+0000 7fe1967fc700 1 -- 172.21.15.144:0/1161700557 learned_addr learned my addr 172.21.15.144:0/1161700557 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1977fe700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe198161340 0x7fe1981597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe196ffd700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1980d2320 0x7fe1981522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1967fc700 1 -- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1980d2320 msgr2=0x7fe1981522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1967fc700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1980d2320 0x7fe1981522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1967fc700 1 -- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe198161340 msgr2=0x7fe1981597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1967fc700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe198161340 0x7fe1981597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1967fc700 1 -- 172.21.15.144:0/1161700557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190009a70 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe1967fc700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 0x7fe1981592b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe18800d800 tx=0x7fe18800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880125f0 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188012750 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1980d15e0 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188012b00 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.321+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1980d1300 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.322+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19815cd80 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.322+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe188012c60 con 0x7fe19815c9b0 2023-11-28T11:53:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.323+0000 7fe17f7fe700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe1840682f0 0x7fe18406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:53:59.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.323+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1880861b0 con 0x7fe19815c9b0 2023-11-28T11:53:59.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.323+0000 7fe196ffd700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe1840682f0 0x7fe18406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:53:59.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.323+0000 7fe196ffd700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe1840682f0 0x7fe18406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe198159ec0 tx=0x7fe18c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:53:59.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.327+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe188052a10 con 0x7fe19815c9b0 2023-11-28T11:53:59.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.556+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe198151630 con 0x7fe19815c9b0 2023-11-28T11:53:59.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.556+0000 7fe17f7fe700 1 -- 172.21.15.144:0/1161700557 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe198151630 con 0x7fe19815c9b0 2023-11-28T11:53:59.559 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:53:59.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe1840682f0 msgr2=0x7fe18406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe1840682f0 0x7fe18406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe198159ec0 tx=0x7fe18c009040 comp rx=0 tx=0).stop 2023-11-28T11:53:59.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 msgr2=0x7fe1981592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:53:59.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 0x7fe1981592b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe18800d800 tx=0x7fe18800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 shutdown_connections 2023-11-28T11:53:59.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe1840682f0 0x7fe18406a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe198161340 0x7fe1981597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1980d2320 0x7fe1981522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 --2- 172.21.15.144:0/1161700557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe19815c9b0 0x7fe1981592b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:53:59.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 >> 172.21.15.144:0/1161700557 conn(0x7fe198072600 msgr2=0x7fe1980b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:53:59.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 shutdown_connections 2023-11-28T11:53:59.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:53:59.558+0000 7fe19d473700 1 -- 172.21.15.144:0/1161700557 wait complete. 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:53:59.572 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:53:59.573 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:53:59.573 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:53:59.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:59.573 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-11-28T11:53:59.590 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:59.591 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42262sec preferred_lft 42262sec 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1121:9a48:f2a:7a1a/64 scope link noprefixroute 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1a:fa:92:22:2e:a9 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.592 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::fccf:aaff:fe54:493c/64 scope link 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether ee:83:c8:ce:35:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec83:c8ff:fece:355d/64 scope link 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1a:fa:92:22:2e:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::18fa:92ff:fe22:2ea9/64 scope link 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:53:59.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:59.594 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:53:59.655 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T11:53:59.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:59.657 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-11-28T11:53:59.716 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-11-28T11:53:59.716 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:53:59.716 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:53:59.716 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:53:59.716 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:53:59.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:59.717 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-11-28T11:53:59.717 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-11-28T11:53:59.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:53:59.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:53:59.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:53:59.717 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-11-28T11:53:59.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:59.821 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:53:59.837 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:53:59.837 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:53:59.837 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:53:59.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:53:59.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-11-28T11:53:59.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:53:59.904 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:53:59.961 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:53:59.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:53:59.961 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:54:00.018 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:54:00.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:54:00.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:54:00.104 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:54:00.104 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:54:00.104 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:54:00.113 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:54:00.113 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:54:00.114 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:54:00.114 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:54:00.114 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:invalid new device string format 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:3300,172.21.15.177:3300,172.21.15.177:3301" 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790" 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.144:6789,172.21.15.177:6789,172.21.15.177:6790:/ 2023-11-28T11:54:00.144 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=af726c13-c3f1-4279-9fca-12c6e4621b18" will pass to kernel 2023-11-28T11:54:00.147 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:54:00.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:54:00.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:54:00.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:54:00.181 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:54:00.262 INFO:teuthology.orchestra.run.smithi144.stdout:client6147 2023-11-28T11:54:00.263 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:54:00.263 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:54:00.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:54:00.320 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T11:54:00.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:54:00.383 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T11:54:00.441 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:54:00.441 DEBUG:teuthology.orchestra.run.smithi144:> import os 2023-11-28T11:54:00.441 DEBUG:teuthology.orchestra.run.smithi144:> import stat 2023-11-28T11:54:00.441 DEBUG:teuthology.orchestra.run.smithi144:> 2023-11-28T11:54:00.441 DEBUG:teuthology.orchestra.run.smithi144:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T11:54:00.441 DEBUG:teuthology.orchestra.run.smithi144:> ' 2023-11-28T11:54:00.547 INFO:teuthology.orchestra.run.smithi144.stdout:1099511627777 2023-11-28T11:54:00.736 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:54:00.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.895+0000 7fc80ce81700 1 Processor -- start 2023-11-28T11:54:00.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.895+0000 7fc80ce81700 1 -- start start 2023-11-28T11:54:00.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80ce81700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 0x7fc808140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80ce81700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc80806f850 0x7fc80806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80ce81700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80813ff00 con 0x7fc8080cdc90 2023-11-28T11:54:00.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80ce81700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8081404c0 con 0x7fc80806f850 2023-11-28T11:54:00.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80ce81700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80815aa80 con 0x7fc8080cd8c0 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc805d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc8080cdc90 0x7fc8080b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55302/0 (socket says 172.21.15.144:55302) 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc805d9b700 1 -- 172.21.15.144:0/3574384363 learned_addr learned my addr 172.21.15.144:0/3574384363 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc806d9d700 1 --2- 172.21.15.144:0/3574384363 >> v2:172.21.15.177:3300/0 conn(0x7fc80806f850 0x7fc80806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80659c700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 0x7fc808140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116547770 0 0) 0x7fc80813ff00 con 0x7fc8080cdc90 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f00035b0 con 0x7fc8080cdc90 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047699259 0 0) 0x7fc7f00035b0 con 0x7fc8080cdc90 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 msgr2=0x7fc808140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 0x7fc808140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 >> v2:172.21.15.177:3300/0 conn(0x7fc80806f850 msgr2=0x7fc80806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 --2- 172.21.15.144:0/3574384363 >> v2:172.21.15.177:3300/0 conn(0x7fc80806f850 0x7fc80806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080b3040 con 0x7fc8080cdc90 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.896+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7fc7fc001670 con 0x7fc8080cdc90 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 >> v1:172.21.15.144:6789/0 conn(0x7fc8080cdc90 legacy=0x7fc8080b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 0x7fc7f0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc7f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7f0004d90 0x7fc7f0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8081404c0 con 0x7fc7f0003f80 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80815aa80 con 0x7fc7f0004d90 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f0009a00 con 0x7fc8080cd8c0 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc7f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80659c700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 0x7fc7f0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc806d9d700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7f0004d90 0x7fc7f0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 -- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 msgr2=0x7fc7f0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080cd8c0 0x7fc7f0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 -- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7f0004d90 msgr2=0x7fc7f0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7f0004d90 0x7fc7f0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 -- 172.21.15.144:0/3574384363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00035b0 con 0x7fc7f0003f80 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc805d9b700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc7f0004850 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8080cea30 tx=0x7fc7f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb20c7d5bdbfc788 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc007c60 con 0x7fc7f0003f80 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00ce20 con 0x7fc7f0003f80 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.897+0000 7fc80559a700 1 -- 172.21.15.144:0/3574384363 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005ba0 con 0x7fc7f0003f80 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc80ce81700 1 -- 172.21.15.144:0/3574384363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 msgr2=0x7fc7f0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc80ce81700 1 --2- 172.21.15.144:0/3574384363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc7f0004850 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8080cea30 tx=0x7fc7f0005880 comp rx=0 tx=0).stop 2023-11-28T11:54:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc806d9d700 1 -- 172.21.15.144:0/3574384363 reap_dead start 2023-11-28T11:54:00.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc80ce81700 1 -- 172.21.15.144:0/3574384363 shutdown_connections 2023-11-28T11:54:00.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc80ce81700 1 -- 172.21.15.144:0/3574384363 >> 172.21.15.144:0/3574384363 conn(0x7fc80806aed0 msgr2=0x7fc80806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:00.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc80ce81700 1 -- 172.21.15.144:0/3574384363 shutdown_connections 2023-11-28T11:54:00.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.898+0000 7fc80ce81700 1 -- 172.21.15.144:0/3574384363 wait complete. 2023-11-28T11:54:00.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.899+0000 7fc80ce81700 1 Processor -- start 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.899+0000 7fc80ce81700 1 -- start start 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80ce81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc8081f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80ce81700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080c49e0 0x7fc8080c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80ce81700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8080b1790 0x7fc8080b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80ce81700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080b2140 con 0x7fc7f0003f80 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80ce81700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080b40e0 con 0x7fc8080b1790 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80ce81700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080b43c0 con 0x7fc8080c49e0 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc8081f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc8081f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38600/0 (socket says 172.21.15.144:38600) 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 -- 172.21.15.144:0/4218229869 learned_addr learned my addr 172.21.15.144:0/4218229869 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc806d9d700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8080b1790 0x7fc8080b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc805d9b700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080c49e0 0x7fc8080c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 -- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080c49e0 msgr2=0x7fc8080c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080c49e0 0x7fc8080c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 -- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8080b1790 msgr2=0x7fc8080b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8080b1790 0x7fc8080b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc80659c700 1 -- 172.21.15.144:0/4218229869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080b35f0 con 0x7fc7f0003f80 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.900+0000 7fc805d9b700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080c49e0 0x7fc8080c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.901+0000 7fc80659c700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc8081f0f70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80119f0 tx=0x7fc7f8011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.901+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8005c30 con 0x7fc7f0003f80 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.901+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8005d90 con 0x7fc7f0003f80 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.901+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800f950 con 0x7fc7f0003f80 2023-11-28T11:54:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.901+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8080b3310 con 0x7fc7f0003f80 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.901+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc808140150 con 0x7fc7f0003f80 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.902+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808068690 con 0x7fc7f0003f80 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.902+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc7f800fab0 con 0x7fc7f0003f80 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.902+0000 7fc7f6ffd700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc7e8068320 0x7fc7e806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.902+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7f80863a0 con 0x7fc7f0003f80 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.903+0000 7fc805d9b700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc7e8068320 0x7fc7e806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.903+0000 7fc805d9b700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc7e8068320 0x7fc7e806a7d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc015040 tx=0x7fc7fc00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:00.907+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7f8052cc0 con 0x7fc7f0003f80 2023-11-28T11:54:01.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.133+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8080cf490 con 0x7fc7f0003f80 2023-11-28T11:54:01.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.133+0000 7fc7f6ffd700 1 -- 172.21.15.144:0/4218229869 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fc7f80562e0 con 0x7fc7f0003f80 2023-11-28T11:54:01.135 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:01.135 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:53:56.350260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:54:01.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc7e8068320 msgr2=0x7fc7e806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc7e8068320 0x7fc7e806a7d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc015040 tx=0x7fc7fc00a000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 msgr2=0x7fc8081f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc8081f0f70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80119f0 tx=0x7fc7f8011d00 comp rx=0 tx=0).stop 2023-11-28T11:54:01.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 shutdown_connections 2023-11-28T11:54:01.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7f0003f80 0x7fc8081f0f70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc7e8068320 0x7fc7e806a7d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8080b1790 0x7fc8080b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 --2- 172.21.15.144:0/4218229869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc8080c49e0 0x7fc8080c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 >> 172.21.15.144:0/4218229869 conn(0x7fc80806aed0 msgr2=0x7fc80813f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 shutdown_connections 2023-11-28T11:54:01.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.135+0000 7fc80ce81700 1 -- 172.21.15.144:0/4218229869 wait complete. 2023-11-28T11:54:01.139 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 31 2023-11-28T11:54:01.150 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T11:54:01.294 INFO:teuthology.orchestra.run.smithi144.stdout:{"message":"","return_code":0} 2023-11-28T11:54:01.295 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:54:01.295 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:54:01.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.336+0000 7f39bb365080 1 Processor -- start 2023-11-28T11:54:01.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.336+0000 7f39bb365080 1 -- start start 2023-11-28T11:54:01.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39bb365080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d6915f0 0x55b53d760640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39bb365080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 0x55b53d6d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39bb365080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d765300 0x55b53d6d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39bb365080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53d638000 con 0x55b53d6d6f40 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39bb365080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53d5cafe0 con 0x55b53d765300 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39bb365080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b53d6c0680 con 0x55b53d6915f0 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39b0c67700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 0x55b53d6d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39b0c67700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 0x55b53d6d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38618/0 (socket says 172.21.15.144:38618) 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39b0c67700 1 -- 172.21.15.144:0/3943551993 learned_addr learned my addr 172.21.15.144:0/3943551993 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39b1c69700 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d765300 0x55b53d6d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.337+0000 7f39b1468700 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d6915f0 0x55b53d760640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b0c67700 1 -- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d6915f0 msgr2=0x55b53d760640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b0c67700 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d6915f0 0x55b53d760640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b0c67700 1 -- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d765300 msgr2=0x55b53d6d4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b0c67700 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d765300 0x55b53d6d4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b0c67700 1 -- 172.21.15.144:0/3943551993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac00a100 con 0x55b53d6d6f40 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b1c69700 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d765300 0x55b53d6d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39b0c67700 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 0x55b53d6d4350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f39a8001dd0 tx=0x7f39a800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed60d9632d4b4c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39a37fe700 1 -- 172.21.15.144:0/3943551993 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8014070 con 0x55b53d6d6f40 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39a37fe700 1 -- 172.21.15.144:0/3943551993 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8008030 con 0x55b53d6d6f40 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39a37fe700 1 -- 172.21.15.144:0/3943551993 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800be50 con 0x55b53d6d6f40 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 -- 172.21.15.144:0/3943551993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 msgr2=0x55b53d6d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 0x55b53d6d4350 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f39a8001dd0 tx=0x7f39a800b4c0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 -- 172.21.15.144:0/3943551993 shutdown_connections 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d765300 0x55b53d6d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d6915f0 0x55b53d760640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 --2- 172.21.15.144:0/3943551993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6d6f40 0x55b53d6d4350 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 -- 172.21.15.144:0/3943551993 >> 172.21.15.144:0/3943551993 conn(0x55b53d6bcb60 msgr2=0x55b53d6d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 -- 172.21.15.144:0/3943551993 shutdown_connections 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.338+0000 7f39bb365080 1 -- 172.21.15.144:0/3943551993 wait complete. 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.339+0000 7f39bb365080 1 Processor -- start 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.339+0000 7f39bb365080 1 -- start start 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6915f0 0x55b53d6b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d6d6f40 0x55b53d75cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d765300 0x55b53d760c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53d6d00c0 con 0x55b53d6915f0 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b53d6b9e10 con 0x55b53d6d6f40 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b53d6bd0d0 con 0x55b53d765300 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6915f0 0x55b53d6b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6915f0 0x55b53d6b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38622/0 (socket says 172.21.15.144:38622) 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 -- 172.21.15.144:0/2569177617 learned_addr learned my addr 172.21.15.144:0/2569177617 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b0c67700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d6d6f40 0x55b53d75cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1c69700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d765300 0x55b53d760c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 -- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d765300 msgr2=0x55b53d760c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b53d765300 0x55b53d760c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 -- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d6d6f40 msgr2=0x55b53d75cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d6d6f40 0x55b53d75cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 -- 172.21.15.144:0/2569177617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a800a040 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b0c67700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53d6d6f40 0x55b53d75cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39b1468700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6915f0 0x55b53d6b1e80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f39a400df00 tx=0x7f39a400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39a27fc700 1 -- 172.21.15.144:0/2569177617 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a400be30 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39a27fc700 1 -- 172.21.15.144:0/2569177617 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4010410 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b53d5d6830 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39a27fc700 1 -- 172.21.15.144:0/2569177617 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a40107c0 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.340+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b53d5e3360 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39a27fc700 1 -- 172.21.15.144:0/2569177617 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39a4020070 con 0x55b53d6915f0 2023-11-28T11:54:01.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39a27fc700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3994066c10 0x7f39940690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39a27fc700 1 -- 172.21.15.144:0/2569177617 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39a400a040 con 0x55b53d6915f0 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39b0c67700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3994066c10 0x7f39940690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39bb365080 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55b53d75d3e0 0x55b53d761180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 --> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60) v8 -- 0x55b53d6d04e0 con 0x55b53d75d3e0 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39b0c67700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3994066c10 0x7f39940690c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55b53d690600 tx=0x7f39a8009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.342+0000 7f39b1c69700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55b53d75d3e0 0x55b53d761180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39b1c69700 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55b53d75d3e0 0x55b53d761180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39b1c69700 1 -- 172.21.15.144:0/2569177617 <== osd.6 v2:172.21.15.177:6800/103771 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f39ac00fb00 con 0x55b53d75d3e0 2023-11-28T11:54:01.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55b53d75d3e0 msgr2=0x55b53d761180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55b53d75d3e0 0x55b53d761180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3994066c10 msgr2=0x7f39940690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3994066c10 0x7f39940690c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55b53d690600 tx=0x7f39a8009000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6915f0 msgr2=0x55b53d6b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 --2- 172.21.15.144:0/2569177617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b53d6915f0 0x55b53d6b1e80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f39a400df00 tx=0x7f39a400b4f0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39b1c69700 1 -- 172.21.15.144:0/2569177617 reap_dead start 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 shutdown_connections 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 >> 172.21.15.144:0/2569177617 conn(0x55b53d6bcb60 msgr2=0x55b53d7649f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 shutdown_connections 2023-11-28T11:54:01.352 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.343+0000 7f39bb365080 1 -- 172.21.15.144:0/2569177617 wait complete. 2023-11-28T11:54:01.353 INFO:teuthology.orchestra.run.smithi144.stdout:Xfiledir_test 2023-11-28T11:54:01.353 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:01.354 DEBUG:teuthology.orchestra.run.smithi144:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 1099511627777, 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: "ancestors": [ 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: "dirino": 1099511627776, 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: "dname": "file", 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: "version": 2 2023-11-28T11:54:01.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: "dirino": 1, 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: "dname": "dir_test", 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: "version": 4 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: "pool": 10, 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: "old_pools": [] 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:54:01.450 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:01.451 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T11:54:01.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 Processor -- start 2023-11-28T11:54:01.536 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 -- start start 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370981f0 0x55f737097c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 0x55f73716fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f73716e320 0x55f7371ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f736fd2fe0 con 0x55f7370c77f0 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f737165fe0 con 0x55f73716e320 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7ee93f080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7370c5830 con 0x55f7370981f0 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 0x55f73716fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 0x55f73716fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38638/0 (socket says 172.21.15.144:38638) 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7dffff700 1 -- 172.21.15.144:0/2547343320 learned_addr learned my addr 172.21.15.144:0/2547343320 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.539 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.535+0000 7fe7e5243700 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f73716e320 0x55f7371ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7e4a42700 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370981f0 0x55f737097c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7dffff700 1 -- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370981f0 msgr2=0x55f737097c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7dffff700 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370981f0 0x55f737097c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7dffff700 1 -- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f73716e320 msgr2=0x55f7371ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7dffff700 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f73716e320 0x55f7371ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7dffff700 1 -- 172.21.15.144:0/2547343320 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d0005de0 con 0x55f7370c77f0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7dffff700 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 0x55f73716fa10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c010 tx=0x7fe7d000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b6a1b0bb89aeaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7deffd700 1 -- 172.21.15.144:0/2547343320 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0008b60 con 0x55f7370c77f0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7deffd700 1 -- 172.21.15.144:0/2547343320 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0008ce0 con 0x55f7370c77f0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7deffd700 1 -- 172.21.15.144:0/2547343320 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001c670 con 0x55f7370c77f0 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2547343320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 msgr2=0x55f73716fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 0x55f73716fa10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c010 tx=0x7fe7d000d000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.540 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2547343320 shutdown_connections 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f73716e320 0x55f7371ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370981f0 0x55f737097c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.536+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2547343320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f7370c77f0 0x55f73716fa10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.537+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2547343320 >> 172.21.15.144:0/2547343320 conn(0x55f7370c4b70 msgr2=0x55f7370c3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.537+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2547343320 shutdown_connections 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.537+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2547343320 wait complete. 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.537+0000 7fe7ee93f080 1 Processor -- start 2023-11-28T11:54:01.541 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.537+0000 7fe7ee93f080 1 -- start start 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7ee93f080 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7370981f0 0x55f737161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7ee93f080 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370c77f0 0x55f73714f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7ee93f080 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f73716e320 0x55f7371661c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7ee93f080 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f737170e50 con 0x55f73716e320 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7ee93f080 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f73716fdb0 con 0x55f7370981f0 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7ee93f080 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f737170090 con 0x55f7370c77f0 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7e5243700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f73716e320 0x55f7371661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7e5243700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f73716e320 0x55f7371661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38644/0 (socket says 172.21.15.144:38644) 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7e5243700 1 -- 172.21.15.144:0/2729900280 learned_addr learned my addr 172.21.15.144:0/2729900280 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7e4a42700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7370981f0 0x55f737161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.544 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.538+0000 7fe7dffff700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370c77f0 0x55f73714f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7e5243700 1 -- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370c77f0 msgr2=0x55f73714f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7e5243700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f7370c77f0 0x55f73714f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7e5243700 1 -- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7370981f0 msgr2=0x55f737161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7e5243700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7370981f0 0x55f737161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7e5243700 1 -- 172.21.15.144:0/2729900280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7370054f0 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7e5243700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f73716e320 0x55f7371661c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0005a30 tx=0x7fe7e000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7ddffb700 1 -- 172.21.15.144:0/2729900280 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00113f0 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7ddffb700 1 -- 172.21.15.144:0/2729900280 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e0011550 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f736feb120 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7ddffb700 1 -- 172.21.15.144:0/2729900280 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0011900 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.539+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f736fde830 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.540+0000 7fe7ddffb700 1 -- 172.21.15.144:0/2729900280 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe7e0021070 con 0x55f73716e320 2023-11-28T11:54:01.545 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7ddffb700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe7c8066c10 0x7fe7c80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7ddffb700 1 -- 172.21.15.144:0/2729900280 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe7e0084940 con 0x55f73716e320 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7e4a42700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe7c8066c10 0x7fe7c80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55f737160cf0 0x55f737210930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 --> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60) v8 -- 0x55f737210e70 con 0x55f737160cf0 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7e4a42700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe7c8066c10 0x7fe7c80690c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800c660 tx=0x7fe7d800e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.541+0000 7fe7dffff700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55f737160cf0 0x55f737210930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.546 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.542+0000 7fe7dffff700 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55f737160cf0 0x55f737210930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7dffff700 1 -- 172.21.15.144:0/2729900280 <== osd.6 v2:172.21.15.177:6800/103771 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v60'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe7d0008960 con 0x55f737160cf0 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55f737160cf0 msgr2=0x55f737210930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x55f737160cf0 0x55f737210930 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe7c8066c10 msgr2=0x7fe7c80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe7c8066c10 0x7fe7c80690c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800c660 tx=0x7fe7d800e040 comp rx=0 tx=0).stop 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f73716e320 msgr2=0x55f7371661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 --2- 172.21.15.144:0/2729900280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f73716e320 0x55f7371661c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0005a30 tx=0x7fe7e000d7b0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7e5243700 1 -- 172.21.15.144:0/2729900280 reap_dead start 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 shutdown_connections 2023-11-28T11:54:01.548 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 >> 172.21.15.144:0/2729900280 conn(0x55f7370c4b70 msgr2=0x55f737097270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 shutdown_connections 2023-11-28T11:54:01.549 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.543+0000 7fe7ee93f080 1 -- 172.21.15.144:0/2729900280 wait complete. 2023-11-28T11:54:01.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T11:54:01.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.671+0000 7f7797c49700 1 Processor -- start 2023-11-28T11:54:01.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.671+0000 7f7797c49700 1 -- start start 2023-11-28T11:54:01.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f7797c49700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f77900af060 0x7f7790070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f7797c49700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f779006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f7797c49700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77900713c0 con 0x7f779006a270 2023-11-28T11:54:01.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f7797c49700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779006dee0 con 0x7f77900af060 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f7797c49700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900716a0 con 0x7f77900af430 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77961e6700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f779006a270 0x7f7790160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55366/0 (socket says 172.21.15.144:55366) 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77961e6700 1 -- 172.21.15.144:0/1255857779 learned_addr learned my addr 172.21.15.144:0/1255857779 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77959e5700 1 --2- 172.21.15.144:0/1255857779 >> v2:172.21.15.177:3300/0 conn(0x7f77900af060 0x7f7790070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77951e4700 1 --2- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f779006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77949e3700 1 -- 172.21.15.144:0/1255857779 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452147882 0 0) 0x7f77900713c0 con 0x7f779006a270 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77949e3700 1 -- 172.21.15.144:0/1255857779 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f777c0035b0 con 0x7f779006a270 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77949e3700 1 -- 172.21.15.144:0/1255857779 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 79550689 0 0) 0x7f777c0035b0 con 0x7f779006a270 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77959e5700 1 -- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 msgr2=0x7f779006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77959e5700 1 --2- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f779006d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77959e5700 1 -- 172.21.15.144:0/1255857779 >> v1:172.21.15.144:6789/0 conn(0x7f779006a270 legacy=0x7f7790160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.672+0000 7f77959e5700 1 -- 172.21.15.144:0/1255857779 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7790071980 con 0x7f77900af060 2023-11-28T11:54:01.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f77959e5700 1 --2- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f7790070d50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f778c009840 tx=0x7f778c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66fc5b9de7d6078e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f77949e3700 1 -- 172.21.15.144:0/1255857779 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880019f0 con 0x7f77900af060 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f77949e3700 1 -- 172.21.15.144:0/1255857779 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c004030 con 0x7f77900af060 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f77949e3700 1 -- 172.21.15.144:0/1255857779 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c0043a0 con 0x7f77900af060 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 -- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 msgr2=0x7f7790070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 --2- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f7790070d50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f778c009840 tx=0x7f778c009b50 comp rx=0 tx=0).stop 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 -- 172.21.15.144:0/1255857779 shutdown_connections 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 --2- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f7790070d50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 --2- 172.21.15.144:0/1255857779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f779006d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 -- 172.21.15.144:0/1255857779 >> 172.21.15.144:0/1255857779 conn(0x7f779006b380 msgr2=0x7f779006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.673+0000 7f7797c49700 1 -- 172.21.15.144:0/1255857779 shutdown_connections 2023-11-28T11:54:01.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.674+0000 7f7797c49700 1 -- 172.21.15.144:0/1255857779 wait complete. 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 Processor -- start 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 -- start start 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 0x7f77900b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f779015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f77900c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900c70b0 con 0x7f779006a270 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900b3fa0 con 0x7f77900af060 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f7797c49700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77900b4280 con 0x7f77900af430 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 0x7f77900b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 0x7f77900b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38656/0 (socket says 172.21.15.144:38656) 2023-11-28T11:54:01.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.675+0000 7f77959e5700 1 -- 172.21.15.144:0/426923832 learned_addr learned my addr 172.21.15.144:0/426923832 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77951e4700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f779015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77959e5700 1 -- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 msgr2=0x7f77900c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77961e6700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f77900c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77959e5700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f77900c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77959e5700 1 -- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 msgr2=0x7f779015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77959e5700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f779015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77959e5700 1 -- 172.21.15.144:0/426923832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0094a0 con 0x7f779006a270 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77951e4700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f779015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77959e5700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 0x7f77900b0160 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f778c009b90 tx=0x7f778c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77867fc700 1 -- 172.21.15.144:0/426923832 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c004590 con 0x7f779006a270 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77867fc700 1 -- 172.21.15.144:0/426923832 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c01f070 con 0x7f779006a270 2023-11-28T11:54:01.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7790071f40 con 0x7f779006a270 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f77867fc700 1 -- 172.21.15.144:0/426923832 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c011870 con 0x7f779006a270 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.676+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790071c60 con 0x7f779006a270 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.678+0000 7f77867fc700 1 -- 172.21.15.144:0/426923832 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f778c017020 con 0x7f779006a270 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.678+0000 7f77867fc700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77740682f0 0x7f777406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.678+0000 7f77867fc700 1 -- 172.21.15.144:0/426923832 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f778c021070 con 0x7f779006a270 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.678+0000 7f77951e4700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77740682f0 0x7f777406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.678+0000 7f77951e4700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77740682f0 0x7f777406a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77900b4a30 tx=0x7f7780008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.680+0000 7f77a3dd2b80 1 Processor -- start 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.680+0000 7f77a3dd2b80 1 -- start start 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b28c0 0x559ac7188ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b31d0 0x559ac71ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac7072c70 con 0x559ac71b01e0 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac70e1400 con 0x559ac71b31d0 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ac70da930 con 0x559ac71b28c0 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38664/0 (socket says 172.21.15.144:38664) 2023-11-28T11:54:01.685 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 -- 172.21.15.144:0/2301328135 learned_addr learned my addr 172.21.15.144:0/2301328135 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77959e5700 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b28c0 0x559ac7188ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77961e6700 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b31d0 0x559ac71ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 -- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b28c0 msgr2=0x559ac7188ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b28c0 0x559ac7188ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 -- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b31d0 msgr2=0x559ac71ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b31d0 0x559ac71ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.681+0000 7f77951e4700 1 -- 172.21.15.144:0/2301328135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ac6f5c690 con 0x559ac71b01e0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77951e4700 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71b2c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f77800109c0 tx=0x7f7780013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23262234acb4cbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77737fe700 1 -- 172.21.15.144:0/2301328135 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780015b00 con 0x559ac71b01e0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77737fe700 1 -- 172.21.15.144:0/2301328135 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778001a070 con 0x559ac71b01e0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77737fe700 1 -- 172.21.15.144:0/2301328135 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780015e60 con 0x559ac71b01e0 2023-11-28T11:54:01.686 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/2301328135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 msgr2=0x559ac71b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71b2c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f77800109c0 tx=0x7f7780013000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/2301328135 shutdown_connections 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b31d0 0x559ac71ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b28c0 0x559ac7188ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/2301328135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71b2c90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/2301328135 >> 172.21.15.144:0/2301328135 conn(0x559ac7185e10 msgr2=0x559ac71867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/2301328135 shutdown_connections 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.682+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/2301328135 wait complete. 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.683+0000 7f77a3dd2b80 1 -- start start 2023-11-28T11:54:01.687 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.683+0000 7f77a3dd2b80 1 Processor -- start 2023-11-28T11:54:01.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac6f5e980 con 0x559ac71b01e0 2023-11-28T11:54:01.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac71a45c0 con 0x559ac71b28c0 2023-11-28T11:54:01.691 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ac71c7b30 con 0x559ac71b31d0 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77a3dd2b80 10 client.0 _ll_register_callbacks cb 0x559ac6f802f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38670/0 (socket says 172.21.15.144:38670) 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 -- 172.21.15.144:0/3032159471 learned_addr learned my addr 172.21.15.144:0/3032159471 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77951e4700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77961e6700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 -- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 msgr2=0x559ac71b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 -- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 msgr2=0x559ac71a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.684+0000 7f77959e5700 1 -- 172.21.15.144:0/3032159471 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778000e050 con 0x559ac71b01e0 2023-11-28T11:54:01.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77959e5700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71a49e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f778c009ea0 tx=0x7f778c0244c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc077c7faee24b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f7770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f7770ff9700 1 -- 172.21.15.144:0/3032159471 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c0543b0 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f7770ff9700 1 -- 172.21.15.144:0/3032159471 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f778c05f700 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ac70a81f0 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f7770ff9700 1 -- 172.21.15.144:0/3032159471 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778c05f8c0 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 10 client.6186 fetch_fsmap learned FSMap version 31 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x559ac6f08e40 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f7770ff9700 1 -- 172.21.15.144:0/3032159471 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ac70a81f0 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f7770ff9700 1 -- 172.21.15.144:0/3032159471 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f778c0538f0 con 0x559ac71b01e0 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 10 client.6186 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 10 client.6186 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 20 client.6186 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:01.693 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 4 client.6186 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ac6f5efc0 con 0x559ac71d1860 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.685+0000 7f77951e4700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f7770ff9700 10 client.6186 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77951e4700 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71b1220 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e3e23ec6a91e7c server_cookie=eb7f7a804e8cacb2 in_seq=0 out_seq=0 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f7770ff9700 1 -- 172.21.15.144:0/3032159471 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x559ac6f5efc0 con 0x559ac71d1860 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f7770ff9700 10 client.6186 handle_command_reply: tid=0 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 client.6186 shutdown 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 msgr2=0x559ac71a49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71a49e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f778c009ea0 tx=0x7f778c0244c0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 shutdown_connections 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71b1220 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71a49e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f7770ff9700 0 client.6186 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/3032159471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 >> 172.21.15.144:0/3032159471 conn(0x559ac7185e10 msgr2=0x559ac71872e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 shutdown_connections 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.686+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/3032159471 wait complete. 2023-11-28T11:54:01.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.687+0000 7f77a3dd2b80 20 client.6186 trim_cache size 0 max 16384 2023-11-28T11:54:01.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.703+0000 7f77a3dd2b80 1 Processor -- start 2023-11-28T11:54:01.705 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.703+0000 7f77a3dd2b80 1 -- start start 2023-11-28T11:54:01.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.707 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac71b2090 con 0x559ac71b01e0 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac71a70e0 con 0x559ac71b28c0 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ac71d1670 con 0x559ac71b31d0 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71ef880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38678/0 (socket says 172.21.15.144:38678) 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 -- 172.21.15.144:0/1021393244 learned_addr learned my addr 172.21.15.144:0/1021393244 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 -- 172.21.15.144:0/1021393244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 msgr2=0x559ac71a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 -- 172.21.15.144:0/1021393244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 msgr2=0x559ac71a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.708 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 -- 172.21.15.144:0/1021393244 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ac70a11f0 con 0x559ac71b01e0 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f77959e5700 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71ef880 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f778c087730 tx=0x7f778c05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a25641511080651 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f7773fff700 1 -- 172.21.15.144:0/1021393244 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c06b020 con 0x559ac71b01e0 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f7773fff700 1 -- 172.21.15.144:0/1021393244 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c058e90 con 0x559ac71b01e0 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.704+0000 7f7773fff700 1 -- 172.21.15.144:0/1021393244 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c053e40 con 0x559ac71b01e0 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/1021393244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 msgr2=0x559ac71ef880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71ef880 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f778c087730 tx=0x7f778c05b000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/1021393244 shutdown_connections 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac71ef880 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/1021393244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/1021393244 >> 172.21.15.144:0/1021393244 conn(0x559ac7185e10 msgr2=0x559ac7189ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.709 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/1021393244 shutdown_connections 2023-11-28T11:54:01.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.705+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/1021393244 wait complete. 2023-11-28T11:54:01.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.706+0000 7f77a3dd2b80 1 -- start start 2023-11-28T11:54:01.710 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.706+0000 7f77a3dd2b80 1 Processor -- start 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac7188010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac7233cd0 con 0x559ac71b01e0 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ac71cfd20 con 0x559ac71b28c0 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ac71efb20 con 0x559ac71b31d0 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 10 client.0 _ll_register_callbacks cb 0x559ac6f802f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac7188010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac7188010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38684/0 (socket says 172.21.15.144:38684) 2023-11-28T11:54:01.713 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 -- 172.21.15.144:0/853798710 learned_addr learned my addr 172.21.15.144:0/853798710 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 -- 172.21.15.144:0/853798710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 msgr2=0x559ac71b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 -- 172.21.15.144:0/853798710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 msgr2=0x559ac71b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 -- 172.21.15.144:0/853798710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ac6f08e40 con 0x559ac71b01e0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77959e5700 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac7188010 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f778c057a30 tx=0x7f778c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6945a3e69102085b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f7770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f7770ff9700 1 -- 172.21.15.144:0/853798710 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c05f700 con 0x559ac71b01e0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f7770ff9700 1 -- 172.21.15.144:0/853798710 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f778c05f860 con 0x559ac71b01e0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ac7098210 con 0x559ac71b01e0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f7770ff9700 1 -- 172.21.15.144:0/853798710 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778c056a60 con 0x559ac71b01e0 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 10 client.6192 fetch_fsmap learned FSMap version 31 2023-11-28T11:54:01.714 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x559ac6f2f2b0 con 0x559ac71b01e0 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f7770ff9700 1 -- 172.21.15.144:0/853798710 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ac7098210 con 0x559ac71b01e0 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.707+0000 7f7770ff9700 1 -- 172.21.15.144:0/853798710 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f778c056bc0 con 0x559ac71b01e0 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77a3dd2b80 10 client.6192 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77a3dd2b80 10 client.6192 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77a3dd2b80 20 client.6192 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77a3dd2b80 4 client.6192 mds_command: new command op to 5988 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x559ac7082220 con 0x559ac71d1860 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77951e4700 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f7770ff9700 10 client.6192 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.708+0000 7f77951e4700 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71a5c10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cbfd8375b1b4276 server_cookie=de5657c873e46e02 in_seq=0 out_seq=0 2023-11-28T11:54:01.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f7770ff9700 1 -- 172.21.15.144:0/853798710 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559ac7082220 con 0x559ac71d1860 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f7770ff9700 10 client.6192 handle_command_reply: tid=0 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 client.6192 shutdown 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 msgr2=0x559ac7188010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac7188010 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f778c057a30 tx=0x7f778c000bc0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 shutdown_connections 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x559ac71d1860 0x559ac71a5c10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ac71b01e0 0x559ac7188010 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559ac71b31d0 0x559ac71b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f7770ff9700 0 client.6192 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 --2- 172.21.15.144:0/853798710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac71b28c0 0x559ac71b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 >> 172.21.15.144:0/853798710 conn(0x559ac7185e10 msgr2=0x559ac7186430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.716 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 shutdown_connections 2023-11-28T11:54:01.717 INFO:tasks.ceph.mds.a.smithi144.stderr:2023-11-28T11:54:01.709+0000 7ff9a8fdc700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6147=0-4194304@1} caps={6147=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6147 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55be786a1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:54:01.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 1 -- 172.21.15.144:0/853798710 wait complete. 2023-11-28T11:54:01.717 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.709+0000 7f77a3dd2b80 20 client.6192 trim_cache size 0 max 16384 2023-11-28T11:54:01.717 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:54:01.718 INFO:teuthology.orchestra.run.smithi144.stdout: "return_code": 0, 2023-11-28T11:54:01.718 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_tag": "d39f759e-324d-4b79-8ae9-271ddf449fae", 2023-11-28T11:54:01.718 INFO:teuthology.orchestra.run.smithi144.stdout: "mode": "asynchronous" 2023-11-28T11:54:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77740682f0 msgr2=0x7f777406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77740682f0 0x7f777406a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77900b4a30 tx=0x7f7780008040 comp rx=0 tx=0).stop 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 msgr2=0x7f77900b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 0x7f77900b0160 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f778c009b90 tx=0x7f778c004750 comp rx=0 tx=0).stop 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 shutdown_connections 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779006a270 0x7f77900b0160 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f77740682f0 0x7f777406a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f77900af430 0x7f77900c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 --2- 172.21.15.144:0/426923832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77900af060 0x7f779015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 >> 172.21.15.144:0/426923832 conn(0x7f779006b380 msgr2=0x7f77900d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 shutdown_connections 2023-11-28T11:54:01.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.710+0000 7f7797c49700 1 -- 172.21.15.144:0/426923832 wait complete. 2023-11-28T11:54:01.724 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:54:01.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.844+0000 7f9ad2b1a700 1 Processor -- start 2023-11-28T11:54:01.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.844+0000 7f9ad2b1a700 1 -- start start 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad2b1a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9acc0ce040 0x7f9acc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad2b1a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9acc0b2930 0x7f9acc0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad2b1a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc06b420 con 0x7f9acc0d0a00 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad2b1a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0b32f0 con 0x7f9acc0b2930 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad2b1a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc06ff60 con 0x7f9acc0ce040 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad10b7700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9acc0b2930 0x7f9acc0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9acbfff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9acc0d0a00 0x7f9acc06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55404/0 (socket says 172.21.15.144:55404) 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9acbfff700 1 -- 172.21.15.144:0/1417427039 learned_addr learned my addr 172.21.15.144:0/1417427039 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:01.849 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/1417427039 >> v2:172.21.15.177:3300/0 conn(0x7f9acc0ce040 0x7f9acc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1417427039 >> v2:172.21.15.177:3300/0 conn(0x7f9acc0ce040 msgr2=0x7f9acc15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/1417427039 >> v2:172.21.15.177:3300/0 conn(0x7f9acc0ce040 0x7f9acc15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1417427039 >> v1:172.21.15.144:6789/0 conn(0x7f9acc0d0a00 legacy=0x7f9acc06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1417427039 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc06b700 con 0x7f9acc0b2930 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9acb7fe700 1 -- 172.21.15.144:0/1417427039 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915653180 0 0) 0x7f9acc06b420 con 0x7f9acc0d0a00 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.845+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/1417427039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b2db0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9acc06eab0 tx=0x7f9ac400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1993cce2075f27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9acb7fe700 1 -- 172.21.15.144:0/1417427039 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e070 con 0x7f9acc0b2930 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9acb7fe700 1 -- 172.21.15.144:0/1417427039 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400bbd0 con 0x7f9acc0b2930 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9acb7fe700 1 -- 172.21.15.144:0/1417427039 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4012430 con 0x7f9acc0b2930 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/1417427039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 msgr2=0x7f9acc0b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.850 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/1417427039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b2db0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9acc06eab0 tx=0x7f9ac400b330 comp rx=0 tx=0).stop 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/1417427039 shutdown_connections 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/1417427039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b2db0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/1417427039 >> v2:172.21.15.177:3300/0 conn(0x7f9acc0ce040 0x7f9acc15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/1417427039 >> 172.21.15.144:0/1417427039 conn(0x7f9acc06aed0 msgr2=0x7f9acc06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/1417427039 shutdown_connections 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.846+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/1417427039 wait complete. 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.847+0000 7f9ad2b1a700 1 Processor -- start 2023-11-28T11:54:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.847+0000 7f9ad2b1a700 1 -- start start 2023-11-28T11:54:01.853 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad2b1a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad2b1a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9acc0ce040 0x7f9acc14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad2b1a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9acc0d0a00 0x7f9acc15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad2b1a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0c52f0 con 0x7f9acc0b2930 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad2b1a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0cac80 con 0x7f9acc0ce040 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad2b1a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0c49c0 con 0x7f9acc0d0a00 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad08b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad08b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38696/0 (socket says 172.21.15.144:38696) 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad08b6700 1 -- 172.21.15.144:0/434456714 learned_addr learned my addr 172.21.15.144:0/434456714 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9acbfff700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9acc0ce040 0x7f9acc14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.848+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9acc0d0a00 0x7f9acc15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad08b6700 1 -- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9acc0d0a00 msgr2=0x7f9acc15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9acc0d0a00 0x7f9acc15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad08b6700 1 -- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9acc0ce040 msgr2=0x7f9acc14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9acc0ce040 0x7f9acc14cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad08b6700 1 -- 172.21.15.144:0/434456714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009a70 con 0x7f9acc0b2930 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9acbfff700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9acc0ce040 0x7f9acc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b3d00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d800 tx=0x7f9ac000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ac97fa700 1 -- 172.21.15.144:0/434456714 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0018070 con 0x7f9acc0b2930 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ac97fa700 1 -- 172.21.15.144:0/434456714 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0005a20 con 0x7f9acc0b2930 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc06bcc0 con 0x7f9acc0b2930 2023-11-28T11:54:01.855 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ac97fa700 1 -- 172.21.15.144:0/434456714 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0005bb0 con 0x7f9acc0b2930 2023-11-28T11:54:01.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.849+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc06b9e0 con 0x7f9acc0b2930 2023-11-28T11:54:01.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.851+0000 7f9ac97fa700 1 -- 172.21.15.144:0/434456714 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ac00125f0 con 0x7f9acc0b2930 2023-11-28T11:54:01.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.851+0000 7f9ac97fa700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ab00682f0 0x7f9ab006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.851+0000 7f9ac97fa700 1 -- 172.21.15.144:0/434456714 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ac0085a40 con 0x7f9acc0b2930 2023-11-28T11:54:01.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.851+0000 7f9acbfff700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ab00682f0 0x7f9ab006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.856 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.851+0000 7f9acbfff700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ab00682f0 0x7f9ab006a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9acc15bb60 tx=0x7f9ab8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 Processor -- start 2023-11-28T11:54:01.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 -- start start 2023-11-28T11:54:01.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b3224fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b32279c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b322957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32139c10 con 0x560b3227a170 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b321a83a0 con 0x560b32277180 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b321a18d0 con 0x560b32279860 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9ad10b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b322957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9ad10b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b322957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38708/0 (socket says 172.21.15.144:38708) 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.853+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1494431114 learned_addr learned my addr 172.21.15.144:0/1494431114 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9acbfff700 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b32279c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b3224fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 msgr2=0x560b3224fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b3224fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 msgr2=0x560b32279c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b32279c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad10b7700 1 -- 172.21.15.144:0/1494431114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b32023630 con 0x560b3227a170 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b322957c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b3aa0 tx=0x7f9ac4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4a09f72483fc8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ab67fc700 1 -- 172.21.15.144:0/1494431114 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4003aa0 con 0x560b3227a170 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ab67fc700 1 -- 172.21.15.144:0/1494431114 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac4003c00 con 0x560b3227a170 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.854+0000 7f9ab67fc700 1 -- 172.21.15.144:0/1494431114 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40125e0 con 0x560b3227a170 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1494431114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 msgr2=0x560b322957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b322957c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b3aa0 tx=0x7f9ac4010000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1494431114 shutdown_connections 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b322957c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.864 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b3224fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1494431114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b32279c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1494431114 >> 172.21.15.144:0/1494431114 conn(0x560b3224cdb0 msgr2=0x560b3224d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1494431114 shutdown_connections 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1494431114 wait complete. 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.855+0000 7f9adeca3b80 1 -- start start 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 Processor -- start 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b3226c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b32268840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32268d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32025920 con 0x560b3227a170 2023-11-28T11:54:01.865 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3226b660 con 0x560b32277180 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b3226bd90 con 0x560b32279860 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9adeca3b80 10 client.0 _ll_register_callbacks cb 0x560b32047290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9ad10b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32268d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9ad10b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32268d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38714/0 (socket says 172.21.15.144:38714) 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.856+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2416856409 learned_addr learned my addr 172.21.15.144:0/2416856409 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b3226c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 msgr2=0x560b32268840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9acbfff700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b32268840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b32268840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 msgr2=0x560b3226c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b3226c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.866 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2416856409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc06b700 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b3226c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32268d80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009fd0 tx=0x7f9ac400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efebdd0b0c877f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9aab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2416856409 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4003ea0 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2416856409 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ac4012cb0 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b3216f190 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2416856409 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ac4012e70 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 10 client.6207 fetch_fsmap learned FSMap version 31 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x560b31fcfde0 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2416856409 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b3216f190 con 0x560b3227a170 2023-11-28T11:54:01.867 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2416856409 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9ac4004040 con 0x560b3227a170 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 10 client.6207 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 10 client.6207 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 20 client.6207 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322683b0 0x560b3227b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 4 client.6207 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.857+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b32025f60 con 0x560b322683b0 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.858+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322683b0 0x560b3227b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.858+0000 7f9aab7fe700 10 client.6207 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.858+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322683b0 0x560b3227b0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89adb2cd2eb6ef06 server_cookie=7b66374c79d9e86f in_seq=0 out_seq=0 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.858+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2416856409 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560b32025f60 con 0x560b322683b0 2023-11-28T11:54:01.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.858+0000 7f9aab7fe700 10 client.6207 handle_command_reply: tid=0 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.858+0000 7f9adeca3b80 1 client.6207 shutdown 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 msgr2=0x560b32268d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32268d80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009fd0 tx=0x7f9ac400e000 comp rx=0 tx=0).stop 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 shutdown_connections 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322683b0 0x560b3227b0c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32268d80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b3226c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2416856409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b32268840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9aab7fe700 0 client.6207 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 >> 172.21.15.144:0/2416856409 conn(0x560b3224cdb0 msgr2=0x560b3224e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 shutdown_connections 2023-11-28T11:54:01.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2416856409 wait complete. 2023-11-28T11:54:01.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.859+0000 7f9adeca3b80 20 client.6207 trim_cache size 0 max 16384 2023-11-28T11:54:01.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.875+0000 7f9adeca3b80 1 Processor -- start 2023-11-28T11:54:01.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.875+0000 7f9adeca3b80 1 -- start start 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 0x560b322976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b3227a170 0x560b3226c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b322659a0 con 0x560b32279860 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32269fe0 con 0x560b32277180 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b32265750 con 0x560b3227a170 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 0x560b322976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.879 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 0x560b322976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38720/0 (socket says 172.21.15.144:38720) 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 -- 172.21.15.144:0/1985688828 learned_addr learned my addr 172.21.15.144:0/1985688828 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 -- 172.21.15.144:0/1985688828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b3227a170 msgr2=0x560b3226c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b3227a170 0x560b3226c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 -- 172.21.15.144:0/1985688828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 msgr2=0x560b322509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 -- 172.21.15.144:0/1985688828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b32168190 con 0x560b32279860 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9acbfff700 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 0x560b322976d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9acc15eec0 tx=0x7f9ab8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a702720688c6c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9ab6ffd700 1 -- 172.21.15.144:0/1985688828 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8005430 con 0x560b32279860 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9ab6ffd700 1 -- 172.21.15.144:0/1985688828 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8005590 con 0x560b32279860 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.876+0000 7f9ab6ffd700 1 -- 172.21.15.144:0/1985688828 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8013420 con 0x560b32279860 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1985688828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 msgr2=0x560b322976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 0x560b322976d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9acc15eec0 tx=0x7f9ab8005f90 comp rx=0 tx=0).stop 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1985688828 shutdown_connections 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b3227a170 0x560b3226c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/1985688828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b32279860 0x560b322976d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1985688828 >> 172.21.15.144:0/1985688828 conn(0x560b3224cdb0 msgr2=0x560b3227d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1985688828 shutdown_connections 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 -- 172.21.15.144:0/1985688828 wait complete. 2023-11-28T11:54:01.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.877+0000 7f9adeca3b80 1 -- start start 2023-11-28T11:54:01.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.878+0000 7f9adeca3b80 1 Processor -- start 2023-11-28T11:54:01.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.878+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.878+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b322489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.878+0000 7f9adeca3b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32266800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3224f0c0 con 0x560b3227a170 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32298510 con 0x560b32277180 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b322e3410 con 0x560b32279860 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 10 client.0 _ll_register_callbacks cb 0x560b32047290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32266800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32266800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38728/0 (socket says 172.21.15.144:38728) 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2991878411 learned_addr learned my addr 172.21.15.144:0/2991878411 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9acbfff700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b322489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 msgr2=0x560b322489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.886 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b322489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 msgr2=0x560b322484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 -- 172.21.15.144:0/2991878411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab800f040 con 0x560b3227a170 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9ad10b7700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32266800 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b730 tx=0x7f9ac4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41d2f061f830363 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9aab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2991878411 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e040 con 0x560b3227a170 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2991878411 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ac4004550 con 0x560b3227a170 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b32279500 con 0x560b3227a170 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2991878411 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ac4004760 con 0x560b3227a170 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 10 client.6213 fetch_fsmap learned FSMap version 31 2023-11-28T11:54:01.887 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x560b31ff6250 con 0x560b3227a170 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2991878411 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b32279500 con 0x560b3227a170 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.879+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2991878411 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 31) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9ac402e440 con 0x560b3227a170 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 10 client.6213 fetch_fsmap finished waiting for FSMap version 31 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 10 client.6213 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 20 client.6213 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322e0120 0x560b32266d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 4 client.6213 mds_command: new command op to 5988 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b3215f1b0 con 0x560b322e0120 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322e0120 0x560b32266d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9aab7fe700 10 client.6213 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9ad08b6700 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322e0120 0x560b32266d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1373ff197b121f7a server_cookie=4d84b235cf8e7bb9 in_seq=0 out_seq=0 2023-11-28T11:54:01.888 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9aab7fe700 1 -- 172.21.15.144:0/2991878411 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x560b3215f1b0 con 0x560b322e0120 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9aab7fe700 10 client.6213 handle_command_reply: tid=0 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 client.6213 shutdown 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 msgr2=0x560b32266800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32266800 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400b730 tx=0x7f9ac4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 shutdown_connections 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x560b322e0120 0x560b32266d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560b3227a170 0x560b32266800 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b32277180 0x560b322484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9aab7fe700 0 client.6213 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 --2- 172.21.15.144:0/2991878411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560b32279860 0x560b322489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.889 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 >> 172.21.15.144:0/2991878411 conn(0x560b3224cdb0 msgr2=0x560b32268a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 shutdown_connections 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.880+0000 7f9adeca3b80 1 -- 172.21.15.144:0/2991878411 wait complete. 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9adeca3b80 20 client.6213 trim_cache size 0 max 16384 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stdout: "scrubs": { 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stdout: "d39f759e-324d-4b79-8ae9-271ddf449fae": { 2023-11-28T11:54:01.890 INFO:teuthology.orchestra.run.smithi144.stdout: "path": "/", 2023-11-28T11:54:01.891 INFO:teuthology.orchestra.run.smithi144.stdout: "tag": "d39f759e-324d-4b79-8ae9-271ddf449fae", 2023-11-28T11:54:01.891 INFO:teuthology.orchestra.run.smithi144.stdout: "options": "recursive" 2023-11-28T11:54:01.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:54:01.891 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:54:01.891 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ab00682f0 msgr2=0x7f9ab006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ab00682f0 0x7f9ab006a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9acc15bb60 tx=0x7f9ab8007d30 comp rx=0 tx=0).stop 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 msgr2=0x7f9acc0b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b3d00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000d800 tx=0x7f9ac000dbc0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 shutdown_connections 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9acc0b2930 0x7f9acc0b3d00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ab00682f0 0x7f9ab006a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9acc0d0a00 0x7f9acc15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.892 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 --2- 172.21.15.144:0/434456714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9acc0ce040 0x7f9acc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:01.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 >> 172.21.15.144:0/434456714 conn(0x7f9acc06aed0 msgr2=0x7f9acc15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:01.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 shutdown_connections 2023-11-28T11:54:01.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:01.881+0000 7f9ad2b1a700 1 -- 172.21.15.144:0/434456714 wait complete. 2023-11-28T11:54:01.895 INFO:tasks.cephfs.filesystem:scrub status for tag:d39f759e-324d-4b79-8ae9-271ddf449fae - {'path': '/', 'tag': 'd39f759e-324d-4b79-8ae9-271ddf449fae', 'options': 'recursive'} 2023-11-28T11:54:05.730 INFO:tasks.ceph.mon.a.smithi144.stderr:2023-11-28T11:54:05.729+0000 7fdb2a043700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T11:54:23.073 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:54:23.077 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:54:31.896 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:54:32.031 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.031+0000 7fa62bcdb700 1 Processor -- start 2023-11-28T11:54:32.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.031+0000 7fa62bcdb700 1 -- start start 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62bcdb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa62406dee0 0x7fa62406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62bcdb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa624160540 0x7fa624160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62bcdb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62415d060 con 0x7fa624071040 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62bcdb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62415d340 con 0x7fa624160540 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62bcdb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62415d620 con 0x7fa62406dee0 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa624160540 0x7fa624160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa624160540 0x7fa624160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38736/0 (socket says 172.21.15.144:38736) 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 -- 172.21.15.144:0/527542656 learned_addr learned my addr 172.21.15.144:0/527542656 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa629276700 1 --2- 172.21.15.144:0/527542656 >> v2:172.21.15.177:3300/0 conn(0x7fa62406dee0 0x7fa62406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 -- 172.21.15.144:0/527542656 >> v2:172.21.15.177:3300/0 conn(0x7fa62406dee0 msgr2=0x7fa62406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 --2- 172.21.15.144:0/527542656 >> v2:172.21.15.177:3300/0 conn(0x7fa62406dee0 0x7fa62406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 -- 172.21.15.144:0/527542656 >> v1:172.21.15.177:6789/0 conn(0x7fa624071040 legacy=0x7fa624071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 -- 172.21.15.144:0/527542656 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62415d900 con 0x7fa624160540 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa62a278700 1 --2- 172.21.15.144:0/527542656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624160980 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa62406fb00 tx=0x7fa61c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e629ebc1fbc62a4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa628a75700 1 -- 172.21.15.144:0/527542656 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00e070 con 0x7fa624160540 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa628a75700 1 -- 172.21.15.144:0/527542656 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c003a20 con 0x7fa624160540 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.032+0000 7fa628a75700 1 -- 172.21.15.144:0/527542656 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c003dd0 con 0x7fa624160540 2023-11-28T11:54:32.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 -- 172.21.15.144:0/527542656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 msgr2=0x7fa624160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/527542656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624160980 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa62406fb00 tx=0x7fa61c00b330 comp rx=0 tx=0).stop 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 -- 172.21.15.144:0/527542656 shutdown_connections 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/527542656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624160980 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/527542656 >> v2:172.21.15.177:3300/0 conn(0x7fa62406dee0 0x7fa62406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 -- 172.21.15.144:0/527542656 >> 172.21.15.144:0/527542656 conn(0x7fa6240d0000 msgr2=0x7fa6240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 -- 172.21.15.144:0/527542656 shutdown_connections 2023-11-28T11:54:32.036 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.033+0000 7fa62bcdb700 1 -- 172.21.15.144:0/527542656 wait complete. 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.035+0000 7fa62bcdb700 1 Processor -- start 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.035+0000 7fa62bcdb700 1 -- start start 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62406dee0 0x7fa624153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa624071040 0x7fa6240ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624155000 con 0x7fa624160540 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa624154380 con 0x7fa624071040 2023-11-28T11:54:32.039 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa624154660 con 0x7fa62406dee0 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38742/0 (socket says 172.21.15.144:38742) 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 -- 172.21.15.144:0/2249044452 learned_addr learned my addr 172.21.15.144:0/2249044452 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa629276700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa624071040 0x7fa6240ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa629a77700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62406dee0 0x7fa624153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 -- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62406dee0 msgr2=0x7fa624153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62406dee0 0x7fa624153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 -- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa624071040 msgr2=0x7fa6240ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa624071040 0x7fa6240ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 -- 172.21.15.144:0/2249044452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c009b30 con 0x7fa624160540 2023-11-28T11:54:32.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa629a77700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62406dee0 0x7fa624153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62a278700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624155400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa62406c7b0 tx=0x7fa61c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa61a7fc700 1 -- 172.21.15.144:0/2249044452 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004440 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa61a7fc700 1 -- 172.21.15.144:0/2249044452 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c0045a0 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62415ccd0 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa61a7fc700 1 -- 172.21.15.144:0/2249044452 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004930 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.036+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62406e870 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.038+0000 7fa61a7fc700 1 -- 172.21.15.144:0/2249044452 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa61c022030 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.038+0000 7fa61a7fc700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6080682f0 0x7fa60806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.038+0000 7fa61a7fc700 1 -- 172.21.15.144:0/2249044452 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa61c087130 con 0x7fa624160540 2023-11-28T11:54:32.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.038+0000 7fa629a77700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6080682f0 0x7fa60806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.039+0000 7fa629a77700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6080682f0 0x7fa60806a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa614006fd0 tx=0x7fa614008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.040+0000 7fa637e64b80 1 Processor -- start 2023-11-28T11:54:32.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.040+0000 7fa637e64b80 1 -- start start 2023-11-28T11:54:32.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906241180 0x555906243c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x55590625f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa637e64b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555906103c10 con 0x555906243860 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559061723a0 con 0x555906244170 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55590616b8d0 con 0x555906241180 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa629a77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa629a77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38746/0 (socket says 172.21.15.144:38746) 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa629a77700 1 -- 172.21.15.144:0/843769227 learned_addr learned my addr 172.21.15.144:0/843769227 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa629276700 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906241180 0x555906243c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.041+0000 7fa62a278700 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x55590625f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa629a77700 1 -- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906241180 msgr2=0x555906243c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa629a77700 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906241180 0x555906243c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa629a77700 1 -- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 msgr2=0x55590625f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.045 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa629a77700 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x55590625f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa629a77700 1 -- 172.21.15.144:0/843769227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555905fed630 con 0x555906243860 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa629a77700 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219a70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa614013970 tx=0x7fa61400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adbccaa9d46c2c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa6077fe700 1 -- 172.21.15.144:0/843769227 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614004220 con 0x555906243860 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa6077fe700 1 -- 172.21.15.144:0/843769227 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa614004380 con 0x555906243860 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa6077fe700 1 -- 172.21.15.144:0/843769227 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614011a60 con 0x555906243860 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa637e64b80 1 -- 172.21.15.144:0/843769227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 msgr2=0x555906219a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.042+0000 7fa637e64b80 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219a70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa614013970 tx=0x7fa61400f000 comp rx=0 tx=0).stop 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 -- 172.21.15.144:0/843769227 shutdown_connections 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219a70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x55590625f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 --2- 172.21.15.144:0/843769227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906241180 0x555906243c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.046 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 -- 172.21.15.144:0/843769227 >> 172.21.15.144:0/843769227 conn(0x555906216db0 msgr2=0x555906217740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 -- 172.21.15.144:0/843769227 shutdown_connections 2023-11-28T11:54:32.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 -- 172.21.15.144:0/843769227 wait complete. 2023-11-28T11:54:32.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 -- start start 2023-11-28T11:54:32.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.043+0000 7fa637e64b80 1 Processor -- start 2023-11-28T11:54:32.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 0x555906235bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x5559062454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555905fef920 con 0x555906241180 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559062357d0 con 0x555906244170 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55590623a260 con 0x555906243860 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa637e64b80 10 client.0 _ll_register_callbacks cb 0x555906011290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa629a77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 0x555906235bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa629a77700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 0x555906235bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38752/0 (socket says 172.21.15.144:38752) 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa629a77700 1 -- 172.21.15.144:0/4015759661 learned_addr learned my addr 172.21.15.144:0/4015759661 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa629276700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.044+0000 7fa62a278700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x5559062454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa629a77700 1 -- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 msgr2=0x5559062625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa629a77700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa629a77700 1 -- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 msgr2=0x5559062454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa629a77700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x5559062454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa629a77700 1 -- 172.21.15.144:0/4015759661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa614013680 con 0x555906241180 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa629a77700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 0x555906235bf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x555906215120 tx=0x7fa6140046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7f985873d8b682 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa604ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa604ff9700 1 -- 172.21.15.144:0/4015759661 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61402a670 con 0x555906241180 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa604ff9700 1 -- 172.21.15.144:0/4015759661 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa61402a7d0 con 0x555906241180 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555906139190 con 0x555906241180 2023-11-28T11:54:32.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa604ff9700 1 -- 172.21.15.144:0/4015759661 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa61402a990 con 0x555906241180 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 10 client.6225 fetch_fsmap learned FSMap version 32 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x555905f99de0 con 0x555906241180 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa604ff9700 1 -- 172.21.15.144:0/4015759661 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555906139190 con 0x555906241180 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa604ff9700 1 -- 172.21.15.144:0/4015759661 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa6140139a0 con 0x555906241180 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 10 client.6225 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 10 client.6225 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 20 client.6225 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062462a0 0x555906241690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 4 client.6225 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.045+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555905feff60 con 0x5559062462a0 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa629276700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062462a0 0x555906241690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa604ff9700 10 client.6225 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa629276700 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062462a0 0x555906241690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12f12de6e4664d1 server_cookie=7dd37383133d77d3 in_seq=0 out_seq=0 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa604ff9700 1 -- 172.21.15.144:0/4015759661 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x555905feff60 con 0x5559062462a0 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa604ff9700 10 client.6225 handle_command_reply: tid=0 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa637e64b80 1 client.6225 shutdown 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 msgr2=0x555906235bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa637e64b80 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 0x555906235bf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x555906215120 tx=0x7fa6140046d0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 shutdown_connections 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.046+0000 7fa637e64b80 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062462a0 0x555906241690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906241180 0x555906235bf0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906244170 0x5559062454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa604ff9700 0 client.6225 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 1 --2- 172.21.15.144:0/4015759661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 >> 172.21.15.144:0/4015759661 conn(0x555906216db0 msgr2=0x555906218280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 shutdown_connections 2023-11-28T11:54:32.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 1 -- 172.21.15.144:0/4015759661 wait complete. 2023-11-28T11:54:32.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.047+0000 7fa637e64b80 20 client.6225 trim_cache size 0 max 16384 2023-11-28T11:54:32.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.063+0000 7fa637e64b80 1 Processor -- start 2023-11-28T11:54:32.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.063+0000 7fa637e64b80 1 -- start start 2023-11-28T11:54:32.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906244170 0x555906241a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa637e64b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555906213f80 con 0x555906243860 2023-11-28T11:54:32.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555906235f90 con 0x555906241180 2023-11-28T11:54:32.066 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x555906258700 con 0x555906244170 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38762/0 (socket says 172.21.15.144:38762) 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 -- 172.21.15.144:0/372894605 learned_addr learned my addr 172.21.15.144:0/372894605 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 -- 172.21.15.144:0/372894605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906244170 msgr2=0x555906241a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906244170 0x555906241a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 -- 172.21.15.144:0/372894605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 msgr2=0x5559062427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 -- 172.21.15.144:0/372894605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555906132190 con 0x555906243860 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa629276700 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x5559062466d0 tx=0x7fa620008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727ab3dd61841433 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa607fff700 1 -- 172.21.15.144:0/372894605 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000ce30 con 0x555906243860 2023-11-28T11:54:32.067 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa607fff700 1 -- 172.21.15.144:0/372894605 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62001b040 con 0x555906243860 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.064+0000 7fa607fff700 1 -- 172.21.15.144:0/372894605 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62001b3f0 con 0x555906243860 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 -- 172.21.15.144:0/372894605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 msgr2=0x555906219720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219720 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x5559062466d0 tx=0x7fa620008040 comp rx=0 tx=0).stop 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 -- 172.21.15.144:0/372894605 shutdown_connections 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906244170 0x555906241a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 --2- 172.21.15.144:0/372894605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906243860 0x555906219720 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 -- 172.21.15.144:0/372894605 >> 172.21.15.144:0/372894605 conn(0x555906216db0 msgr2=0x5559062623e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 -- 172.21.15.144:0/372894605 shutdown_connections 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.065+0000 7fa637e64b80 1 -- 172.21.15.144:0/372894605 wait complete. 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.066+0000 7fa637e64b80 1 -- start start 2023-11-28T11:54:32.068 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.066+0000 7fa637e64b80 1 Processor -- start 2023-11-28T11:54:32.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 0x5559062c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559062c3bf0 con 0x555906244170 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555906247b20 con 0x555906241180 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559062a99c0 con 0x555906243860 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa637e64b80 10 client.0 _ll_register_callbacks cb 0x555906011290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa62a278700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 0x5559062c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa62a278700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 0x5559062c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38770/0 (socket says 172.21.15.144:38770) 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa62a278700 1 -- 172.21.15.144:0/3387312158 learned_addr learned my addr 172.21.15.144:0/3387312158 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa629a77700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.067+0000 7fa629276700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa62a278700 1 -- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 msgr2=0x5559062475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa62a278700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa62a278700 1 -- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 msgr2=0x5559062575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa62a278700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa62a278700 1 -- 172.21.15.144:0/3387312158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62000cb60 con 0x555906244170 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa62a278700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 0x5559062c4010 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009e80 tx=0x7fa61c056e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efa48ae078cbc8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa604ff9700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa604ff9700 1 -- 172.21.15.144:0/3387312158 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c05f2d0 con 0x555906244170 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa604ff9700 1 -- 172.21.15.144:0/3387312158 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fa61c05f560 con 0x555906244170 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559061291b0 con 0x555906244170 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa604ff9700 1 -- 172.21.15.144:0/3387312158 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa61c007310 con 0x555906244170 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 10 client.6231 fetch_fsmap learned FSMap version 32 2023-11-28T11:54:32.074 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x555905fc0250 con 0x555906244170 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa604ff9700 1 -- 172.21.15.144:0/3387312158 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5559061291b0 con 0x555906244170 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa604ff9700 1 -- 172.21.15.144:0/3387312158 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fa61c0532c0 con 0x555906244170 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 10 client.6231 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 10 client.6231 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 20 client.6231 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062616d0 0x5559062124a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 4 client.6231 mds_command: new command op to 5988 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5559061131c0 con 0x5559062616d0 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.068+0000 7fa629a77700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062616d0 0x5559062124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa604ff9700 10 client.6231 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa629a77700 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062616d0 0x5559062124a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675bd9aee0f2ea34 server_cookie=adb8196e6c3da4d3 in_seq=0 out_seq=0 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa604ff9700 1 -- 172.21.15.144:0/3387312158 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5559061131c0 con 0x5559062616d0 2023-11-28T11:54:32.075 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa604ff9700 10 client.6231 handle_command_reply: tid=0 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 client.6231 shutdown 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 msgr2=0x5559062c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 0x5559062c4010 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009e80 tx=0x7fa61c056e30 comp rx=0 tx=0).stop 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 shutdown_connections 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x5559062616d0 0x5559062124a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555906244170 0x5559062c4010 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa604ff9700 0 client.6231 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555906241180 0x5559062575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 --2- 172.21.15.144:0/3387312158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x555906243860 0x5559062475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 >> 172.21.15.144:0/3387312158 conn(0x555906216db0 msgr2=0x555906217420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.076 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 shutdown_connections 2023-11-28T11:54:32.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 1 -- 172.21.15.144:0/3387312158 wait complete. 2023-11-28T11:54:32.077 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.069+0000 7fa637e64b80 20 client.6231 trim_cache size 0 max 16384 2023-11-28T11:54:32.077 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:54:32.077 INFO:teuthology.orchestra.run.smithi144.stdout: "status": "no active scrubs running", 2023-11-28T11:54:32.077 INFO:teuthology.orchestra.run.smithi144.stdout: "scrubs": {} 2023-11-28T11:54:32.077 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6080682f0 msgr2=0x7fa60806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6080682f0 0x7fa60806a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa614006fd0 tx=0x7fa614008040 comp rx=0 tx=0).stop 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 msgr2=0x7fa624155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624155400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa62406c7b0 tx=0x7fa61c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 shutdown_connections 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa624160540 0x7fa624155400 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.078 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa6080682f0 0x7fa60806a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa62406dee0 0x7fa624153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 --2- 172.21.15.144:0/2249044452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa624071040 0x7fa6240ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 >> 172.21.15.144:0/2249044452 conn(0x7fa6240d0000 msgr2=0x7fa62415e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 shutdown_connections 2023-11-28T11:54:32.079 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.070+0000 7fa62bcdb700 1 -- 172.21.15.144:0/2249044452 wait complete. 2023-11-28T11:54:32.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:54:32.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:54:32.202 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.202+0000 7f21acffb700 1 Processor -- start 2023-11-28T11:54:32.203 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.202+0000 7f21acffb700 1 -- start start 2023-11-28T11:54:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21acffb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f21a8160740 0x7f21a80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21acffb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a8160b10 0x7f21a80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21acffb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f21a806b060 0x7f21a806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21acffb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a80ca720 con 0x7f21a806b060 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21acffb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a80cc030 con 0x7f21a8160740 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21acffb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80c9e10 con 0x7f21a8160b10 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f21a806b060 0x7f21a806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f21a806b060 0x7f21a806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38778/0 (socket says 172.21.15.144:38778) 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 -- 172.21.15.144:0/1334623055 learned_addr learned my addr 172.21.15.144:0/1334623055 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a659c700 1 --2- 172.21.15.144:0/1334623055 >> v2:172.21.15.177:3300/0 conn(0x7f21a8160740 0x7f21a80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 -- 172.21.15.144:0/1334623055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a8160b10 msgr2=0x7f21a80cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a5d9b700 1 --2- 172.21.15.144:0/1334623055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a8160b10 0x7f21a80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 --2- 172.21.15.144:0/1334623055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a8160b10 0x7f21a80cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 -- 172.21.15.144:0/1334623055 >> v2:172.21.15.177:3300/0 conn(0x7f21a8160740 msgr2=0x7f21a80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 --2- 172.21.15.144:0/1334623055 >> v2:172.21.15.177:3300/0 conn(0x7f21a8160740 0x7f21a80cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 -- 172.21.15.144:0/1334623055 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a806efc0 con 0x7f21a806b060 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a6d9d700 1 --2- 172.21.15.144:0/1334623055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a806b060 0x7f21a806b4e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f21a80b23b0 tx=0x7f21a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221630cdb563ea2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.203+0000 7f21a559a700 1 -- 172.21.15.144:0/1334623055 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000e070 con 0x7f21a806b060 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21a559a700 1 -- 172.21.15.144:0/1334623055 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a000b9c0 con 0x7f21a806b060 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21a559a700 1 -- 172.21.15.144:0/1334623055 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000bd70 con 0x7f21a806b060 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 -- 172.21.15.144:0/1334623055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a806b060 msgr2=0x7f21a806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 --2- 172.21.15.144:0/1334623055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a806b060 0x7f21a806b4e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f21a80b23b0 tx=0x7f21a000b330 comp rx=0 tx=0).stop 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 -- 172.21.15.144:0/1334623055 shutdown_connections 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 --2- 172.21.15.144:0/1334623055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a806b060 0x7f21a806b4e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 --2- 172.21.15.144:0/1334623055 >> v2:172.21.15.177:3300/0 conn(0x7f21a8160740 0x7f21a80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 --2- 172.21.15.144:0/1334623055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a8160b10 0x7f21a80cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 -- 172.21.15.144:0/1334623055 >> 172.21.15.144:0/1334623055 conn(0x7f21a80b12b0 msgr2=0x7f21a80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 -- 172.21.15.144:0/1334623055 shutdown_connections 2023-11-28T11:54:32.208 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.204+0000 7f21acffb700 1 -- 172.21.15.144:0/1334623055 wait complete. 2023-11-28T11:54:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 Processor -- start 2023-11-28T11:54:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 -- start start 2023-11-28T11:54:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a806b060 0x7f21a81f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a8160740 0x7f21a80c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 0x7f21a8157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a81f6f30 con 0x7f21a8160b10 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a815a820 con 0x7f21a8160740 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21acffb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80c4590 con 0x7f21a806b060 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21a6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 0x7f21a8157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21a6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 0x7f21a8157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38784/0 (socket says 172.21.15.144:38784) 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.206+0000 7f21a6d9d700 1 -- 172.21.15.144:0/3659672799 learned_addr learned my addr 172.21.15.144:0/3659672799 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a659c700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a806b060 0x7f21a81f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a5d9b700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a8160740 0x7f21a80c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a6d9d700 1 -- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a806b060 msgr2=0x7f21a81f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a6d9d700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a806b060 0x7f21a81f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a6d9d700 1 -- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a8160740 msgr2=0x7f21a80c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a6d9d700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a8160740 0x7f21a80c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a6d9d700 1 -- 172.21.15.144:0/3659672799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0009b30 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a5d9b700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a8160740 0x7f21a80c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21a6d9d700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 0x7f21a8157060 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f21a000b370 tx=0x7f21a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004310 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a0004470 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0004820 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a80ca3b0 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.207+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a80ca0d0 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.208+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a80cd8d0 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.208+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f21a001b430 con 0x7f21a8160b10 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.209+0000 7f2196ffd700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f21880682f0 0x7f218806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.209+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f21a0087120 con 0x7f21a8160b10 2023-11-28T11:54:32.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.209+0000 7f21a659c700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f21880682f0 0x7f218806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.209+0000 7f21a659c700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f21880682f0 0x7f218806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f21a8158ef0 tx=0x7f2198008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.213+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f21a0053890 con 0x7f21a8160b10 2023-11-28T11:54:32.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.435+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a8072930 con 0x7f21a8160b10 2023-11-28T11:54:32.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.435+0000 7f2196ffd700 1 -- 172.21.15.144:0/3659672799 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f21a8072930 con 0x7f21a8160b10 2023-11-28T11:54:32.436 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:32.436 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:54:05.730553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f21880682f0 msgr2=0x7f218806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f21880682f0 0x7f218806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f21a8158ef0 tx=0x7f2198008040 comp rx=0 tx=0).stop 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 msgr2=0x7f21a8157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 0x7f21a8157060 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f21a000b370 tx=0x7f21a0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 shutdown_connections 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21a8160b10 0x7f21a8157060 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f21880682f0 0x7f218806a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21a806b060 0x7f21a81f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 --2- 172.21.15.144:0/3659672799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21a8160740 0x7f21a80c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 >> 172.21.15.144:0/3659672799 conn(0x7f21a80b12b0 msgr2=0x7f21a8160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 shutdown_connections 2023-11-28T11:54:32.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.437+0000 7f21acffb700 1 -- 172.21.15.144:0/3659672799 wait complete. 2023-11-28T11:54:32.440 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2023-11-28T11:54:32.450 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-28T11:54:32.572 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 Processor -- start 2023-11-28T11:54:32.573 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 -- start start 2023-11-28T11:54:32.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4608071040 0x7f4608071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f460806dee0 0x7f460806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4608071950 con 0x7f460806cdc0 2023-11-28T11:54:32.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4608161550 con 0x7f460806dee0 2023-11-28T11:54:32.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.572+0000 7f460e2c5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460815d060 con 0x7f4608071040 2023-11-28T11:54:32.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f460806dee0 0x7f460806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f460806dee0 0x7f460806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38788/0 (socket says 172.21.15.144:38788) 2023-11-28T11:54:32.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 -- 172.21.15.144:0/2143269198 learned_addr learned my addr 172.21.15.144:0/2143269198 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f4607fff700 1 --2- 172.21.15.144:0/2143269198 >> v2:172.21.15.177:3300/0 conn(0x7f4608071040 0x7f4608071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 -- 172.21.15.144:0/2143269198 >> v2:172.21.15.177:3300/0 conn(0x7f4608071040 msgr2=0x7f4608071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 --2- 172.21.15.144:0/2143269198 >> v2:172.21.15.177:3300/0 conn(0x7f4608071040 0x7f4608071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 -- 172.21.15.144:0/2143269198 >> v1:172.21.15.177:6789/0 conn(0x7f460806cdc0 legacy=0x7f460806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 -- 172.21.15.144:0/2143269198 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460815d340 con 0x7f460806dee0 2023-11-28T11:54:32.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f46077fe700 1 --2- 172.21.15.144:0/2143269198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806dee0 0x7f460806e330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f46080d0f50 tx=0x7f45fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f062c8ba096f33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f4606ffd700 1 -- 172.21.15.144:0/2143269198 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc004230 con 0x7f460806dee0 2023-11-28T11:54:32.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f4606ffd700 1 -- 172.21.15.144:0/2143269198 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc004390 con 0x7f460806dee0 2023-11-28T11:54:32.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.573+0000 7f4606ffd700 1 -- 172.21.15.144:0/2143269198 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0036a0 con 0x7f460806dee0 2023-11-28T11:54:32.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 -- 172.21.15.144:0/2143269198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806dee0 msgr2=0x7f460806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2143269198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806dee0 0x7f460806e330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f46080d0f50 tx=0x7f45fc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 -- 172.21.15.144:0/2143269198 shutdown_connections 2023-11-28T11:54:32.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2143269198 >> v2:172.21.15.177:3300/0 conn(0x7f4608071040 0x7f4608071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2143269198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806dee0 0x7f460806e330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 -- 172.21.15.144:0/2143269198 >> 172.21.15.144:0/2143269198 conn(0x7f46080d0000 msgr2=0x7f46080d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 -- 172.21.15.144:0/2143269198 shutdown_connections 2023-11-28T11:54:32.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.574+0000 7f460e2c5700 1 -- 172.21.15.144:0/2143269198 wait complete. 2023-11-28T11:54:32.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.575+0000 7f460e2c5700 1 Processor -- start 2023-11-28T11:54:32.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.575+0000 7f460e2c5700 1 -- start start 2023-11-28T11:54:32.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 0x7f46080b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460806dee0 0x7f46081f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4608071040 0x7f46080b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080ae950 con 0x7f460806cdc0 2023-11-28T11:54:32.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080b3160 con 0x7f460806dee0 2023-11-28T11:54:32.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46081f3740 con 0x7f4608071040 2023-11-28T11:54:32.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 0x7f46080b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 0x7f46080b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38794/0 (socket says 172.21.15.144:38794) 2023-11-28T11:54:32.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 -- 172.21.15.144:0/2837648714 learned_addr learned my addr 172.21.15.144:0/2837648714 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.588 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 -- 172.21.15.144:0/2837648714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4608071040 msgr2=0x7f46080b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.588 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4608071040 0x7f46080b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.588 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 -- 172.21.15.144:0/2837648714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460806dee0 msgr2=0x7f46081f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.588 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460806dee0 0x7f46081f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.588 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 -- 172.21.15.144:0/2837648714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc009550 con 0x7f460806cdc0 2023-11-28T11:54:32.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4607fff700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 0x7f46080b0f70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c7f0 tx=0x7f45f800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4604ff9700 1 -- 172.21.15.144:0/2837648714 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8011800 con 0x7f460806cdc0 2023-11-28T11:54:32.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4604ff9700 1 -- 172.21.15.144:0/2837648714 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8011960 con 0x7f460806cdc0 2023-11-28T11:54:32.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460815d900 con 0x7f460806cdc0 2023-11-28T11:54:32.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.576+0000 7f4604ff9700 1 -- 172.21.15.144:0/2837648714 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8011cf0 con 0x7f460806cdc0 2023-11-28T11:54:32.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.577+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460815d620 con 0x7f460806cdc0 2023-11-28T11:54:32.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.578+0000 7f4604ff9700 1 -- 172.21.15.144:0/2837648714 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f45f8021070 con 0x7f460806cdc0 2023-11-28T11:54:32.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.578+0000 7f4604ff9700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45f40682f0 0x7f45f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.578+0000 7f4604ff9700 1 -- 172.21.15.144:0/2837648714 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f45f8085b30 con 0x7f460806cdc0 2023-11-28T11:54:32.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.578+0000 7f46077fe700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45f40682f0 0x7f45f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.579+0000 7f46077fe700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45f40682f0 0x7f45f406a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46080cfc10 tx=0x7f45fc010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.580+0000 7f461a44eb80 1 Processor -- start 2023-11-28T11:54:32.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.580+0000 7f461a44eb80 1 -- start start 2023-11-28T11:54:32.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 0x55d5da224ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24c1d0 0x55d5da24ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da26a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f461a44eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da10ec60 con 0x55d5da24e8b0 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da17d3f0 con 0x55d5da24f1c0 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5da176920 con 0x55d5da24c1d0 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 0x55d5da224ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 0x55d5da224ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38802/0 (socket says 172.21.15.144:38802) 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 -- 172.21.15.144:0/2138906173 learned_addr learned my addr 172.21.15.144:0/2138906173 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f46077fe700 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24c1d0 0x55d5da24ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f460c862700 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da26a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 -- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24c1d0 msgr2=0x55d5da24ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24c1d0 0x55d5da24ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 -- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 msgr2=0x55d5da26a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da26a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.581+0000 7f4607fff700 1 -- 172.21.15.144:0/2138906173 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d9ff8680 con 0x55d5da24e8b0 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f4607fff700 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 0x55d5da224ac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f45f805e120 tx=0x7f45f805e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaa55cb912702c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f45edffb700 1 -- 172.21.15.144:0/2138906173 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8054420 con 0x55d5da24e8b0 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f45edffb700 1 -- 172.21.15.144:0/2138906173 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f806d210 con 0x55d5da24e8b0 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f45edffb700 1 -- 172.21.15.144:0/2138906173 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8051fb0 con 0x55d5da24e8b0 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 -- 172.21.15.144:0/2138906173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 msgr2=0x55d5da224ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 0x55d5da224ac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f45f805e120 tx=0x7f45f805e150 comp rx=0 tx=0).stop 2023-11-28T11:54:32.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 -- 172.21.15.144:0/2138906173 shutdown_connections 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24e8b0 0x55d5da224ac0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da26a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2138906173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24c1d0 0x55d5da24ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 -- 172.21.15.144:0/2138906173 >> 172.21.15.144:0/2138906173 conn(0x55d5da221e00 msgr2=0x55d5da222790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 -- 172.21.15.144:0/2138906173 shutdown_connections 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.582+0000 7f461a44eb80 1 -- 172.21.15.144:0/2138906173 wait complete. 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.583+0000 7f461a44eb80 1 -- start start 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.583+0000 7f461a44eb80 1 Processor -- start 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da24c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24e8b0 0x55d5da240c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24f1c0 0x55d5da23cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d9ffa970 con 0x55d5da24c1d0 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da261bb0 con 0x55d5da24e8b0 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5da24d140 con 0x55d5da24f1c0 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f461a44eb80 10 client.0 _ll_register_callbacks cb 0x55d5da01c2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da24c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da24c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38808/0 (socket says 172.21.15.144:38808) 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 -- 172.21.15.144:0/3318663083 learned_addr learned my addr 172.21.15.144:0/3318663083 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f460c862700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24f1c0 0x55d5da23cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f46077fe700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24e8b0 0x55d5da240c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 -- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24f1c0 msgr2=0x55d5da23cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24f1c0 0x55d5da23cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 -- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24e8b0 msgr2=0x55d5da240c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24e8b0 0x55d5da240c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 -- 172.21.15.144:0/3318663083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460815d620 con 0x55d5da24c1d0 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f460c862700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24f1c0 0x55d5da23cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f4607fff700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da24c650 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f45f80524c0 tx=0x7f45f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fdb0d3b6d568c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f45e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.584+0000 7f45e6ffd700 1 -- 172.21.15.144:0/3318663083 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8069470 con 0x55d5da24c1d0 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f45e6ffd700 1 -- 172.21.15.144:0/3318663083 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f45f8054420 con 0x55d5da24c1d0 2023-11-28T11:54:32.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5da1441e0 con 0x55d5da24c1d0 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f45e6ffd700 1 -- 172.21.15.144:0/3318663083 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f8054f20 con 0x55d5da24c1d0 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 10 client.6249 fetch_fsmap learned FSMap version 32 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55d5d9fa4e30 con 0x55d5da24c1d0 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f45e6ffd700 1 -- 172.21.15.144:0/3318663083 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5da1441e0 con 0x55d5da24c1d0 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f45e6ffd700 1 -- 172.21.15.144:0/3318663083 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f45f8055080 con 0x55d5da24c1d0 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 10 client.6249 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 10 client.6249 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 20 client.6249 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da260cc0 0x55d5da261090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 4 client.6249 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:54:32.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5d9ffafb0 con 0x55d5da260cc0 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f46077fe700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da260cc0 0x55d5da261090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f46077fe700 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da260cc0 0x55d5da261090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4e9823909587d8 server_cookie=16435aeee700ee93 in_seq=0 out_seq=0 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.585+0000 7f45e6ffd700 10 client.6249 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f45e6ffd700 1 -- 172.21.15.144:0/3318663083 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d5d9ffafb0 con 0x55d5da260cc0 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f45e6ffd700 10 client.6249 handle_command_reply: tid=0 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 client.6249 shutdown 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 msgr2=0x55d5da24c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da24c650 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f45f80524c0 tx=0x7f45f8013000 comp rx=0 tx=0).stop 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 shutdown_connections 2023-11-28T11:54:32.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da260cc0 0x55d5da261090 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da24c650 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24f1c0 0x55d5da23cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f45e6ffd700 0 client.6249 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 --2- 172.21.15.144:0/3318663083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24e8b0 0x55d5da240c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 >> 172.21.15.144:0/3318663083 conn(0x55d5da221e00 msgr2=0x55d5da224a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 shutdown_connections 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 1 -- 172.21.15.144:0/3318663083 wait complete. 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.586+0000 7f461a44eb80 20 client.6249 trim_cache size 0 max 16384 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.599+0000 7f461a44eb80 1 Processor -- start 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.599+0000 7f461a44eb80 1 -- start start 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24c1d0 0x55d5da2224c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da222a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 0x55d5da23f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f461a44eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da263240 con 0x55d5da24f1c0 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da26cb60 con 0x55d5da24c1d0 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5da252b30 con 0x55d5da24e8b0 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 0x55d5da23f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 0x55d5da23f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38820/0 (socket says 172.21.15.144:38820) 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 -- 172.21.15.144:0/1616835002 learned_addr learned my addr 172.21.15.144:0/1616835002 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f4607fff700 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24c1d0 0x55d5da2224c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f46077fe700 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da222a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 -- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 msgr2=0x55d5da222a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da222a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 -- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24c1d0 msgr2=0x55d5da2224c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24c1d0 0x55d5da2224c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 -- 172.21.15.144:0/1616835002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5da13d1e0 con 0x55d5da24f1c0 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f460c862700 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 0x55d5da23f940 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4600009ea0 tx=0x7f4600009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce31a584020fc826 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f45ee7fc700 1 -- 172.21.15.144:0/1616835002 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600011040 con 0x55d5da24f1c0 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f45ee7fc700 1 -- 172.21.15.144:0/1616835002 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460000ae40 con 0x55d5da24f1c0 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.600+0000 7f45ee7fc700 1 -- 172.21.15.144:0/1616835002 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600016680 con 0x55d5da24f1c0 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 -- 172.21.15.144:0/1616835002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 msgr2=0x55d5da23f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 0x55d5da23f940 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4600009ea0 tx=0x7f4600009ed0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 -- 172.21.15.144:0/1616835002 shutdown_connections 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24f1c0 0x55d5da23f940 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24c1d0 0x55d5da2224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 --2- 172.21.15.144:0/1616835002 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da222a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 -- 172.21.15.144:0/1616835002 >> 172.21.15.144:0/1616835002 conn(0x55d5da221e00 msgr2=0x55d5da241300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 -- 172.21.15.144:0/1616835002 shutdown_connections 2023-11-28T11:54:32.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 -- 172.21.15.144:0/1616835002 wait complete. 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.601+0000 7f461a44eb80 1 -- start start 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 Processor -- start 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da245c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da2522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da2cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da23d3d0 con 0x55d5da24c1d0 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5da2528a0 con 0x55d5da24f1c0 2023-11-28T11:54:32.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5da245510 con 0x55d5da24e8b0 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.602+0000 7f461a44eb80 10 client.0 _ll_register_callbacks cb 0x55d5da01c2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da245c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da245c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38822/0 (socket says 172.21.15.144:38822) 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 -- 172.21.15.144:0/2289999199 learned_addr learned my addr 172.21.15.144:0/2289999199 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f46077fe700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da2522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f460c862700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da2cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 -- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 msgr2=0x55d5da2522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da2522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 -- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 msgr2=0x55d5da2cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da2cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 -- 172.21.15.144:0/2289999199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4600009b00 con 0x55d5da24c1d0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f460c862700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da2cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f46077fe700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da2522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f4607fff700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da245c80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f45f805db20 tx=0x7f45f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f919a35e238e18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f45e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f45e6ffd700 1 -- 172.21.15.144:0/2289999199 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8055d30 con 0x55d5da24c1d0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f45e6ffd700 1 -- 172.21.15.144:0/2289999199 <== mon.0 v2:172.21.15.144:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f45f8054420 con 0x55d5da24c1d0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5da134200 con 0x55d5da24c1d0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f45e6ffd700 1 -- 172.21.15.144:0/2289999199 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f8052100 con 0x55d5da24c1d0 2023-11-28T11:54:32.622 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 10 client.6255 fetch_fsmap learned FSMap version 32 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55d5d9fcb2a0 con 0x55d5da24c1d0 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f45e6ffd700 1 -- 172.21.15.144:0/2289999199 <== mon.0 v2:172.21.15.144:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d5da134200 con 0x55d5da24c1d0 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f45e6ffd700 1 -- 172.21.15.144:0/2289999199 <== mon.0 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f45f807e080 con 0x55d5da24c1d0 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 10 client.6255 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 10 client.6255 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 20 client.6255 populate_metadata read hostname 'smithi144' 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da24d5f0 0x55d5da23fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 4 client.6255 mds_command: new command op to 5988 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.603+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 --> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55d5da11e210 con 0x55d5da24d5f0 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f46077fe700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da24d5f0 0x55d5da23fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f45e6ffd700 10 client.6255 ms_handle_connect on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.623 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f46077fe700 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da24d5f0 0x55d5da23fc00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ef3e0863e9c44d server_cookie=c5e9501b997295f2 in_seq=0 out_seq=0 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f45e6ffd700 1 -- 172.21.15.144:0/2289999199 <== mds.0 v2:172.21.15.144:6835/3077123372 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55d5da11e210 con 0x55d5da24d5f0 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f45e6ffd700 10 client.6255 handle_command_reply: tid=0 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 client.6255 shutdown 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 msgr2=0x55d5da245c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da245c80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f45f805db20 tx=0x7f45f801c000 comp rx=0 tx=0).stop 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 shutdown_connections 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:6835/3077123372,v1:172.21.15.144:6837/3077123372] conn(0x55d5da24d5f0 0x55d5da23fc00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d5da24c1d0 0x55d5da245c80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f45e6ffd700 0 client.6255 ms_handle_reset on v2:172.21.15.144:6835/3077123372 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d5da24f1c0 0x55d5da2cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 --2- 172.21.15.144:0/2289999199 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55d5da24e8b0 0x55d5da2522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 >> 172.21.15.144:0/2289999199 conn(0x55d5da221e00 msgr2=0x55d5da240920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 shutdown_connections 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 1 -- 172.21.15.144:0/2289999199 wait complete. 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.604+0000 7f461a44eb80 20 client.6255 trim_cache size 0 max 16384 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stdout:[ 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stdout: { 2023-11-28T11:54:32.625 INFO:teuthology.orchestra.run.smithi144.stdout: "damage_type": "backtrace", 2023-11-28T11:54:32.626 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 2511412639, 2023-11-28T11:54:32.626 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 1099511627777, 2023-11-28T11:54:32.626 INFO:teuthology.orchestra.run.smithi144.stdout: "path": "/dir_test/file" 2023-11-28T11:54:32.626 INFO:teuthology.orchestra.run.smithi144.stdout: } 2023-11-28T11:54:32.626 INFO:teuthology.orchestra.run.smithi144.stdout:] 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45f40682f0 msgr2=0x7f45f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45f40682f0 0x7f45f406a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46080cfc10 tx=0x7f45fc010000 comp rx=0 tx=0).stop 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 msgr2=0x7f46080b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 0x7f46080b0f70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c7f0 tx=0x7f45f800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 shutdown_connections 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f460806cdc0 0x7f46080b0f70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f45f40682f0 0x7f45f406a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4608071040 0x7f46080b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 --2- 172.21.15.144:0/2837648714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460806dee0 0x7f46081f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 >> 172.21.15.144:0/2837648714 conn(0x7f46080d0000 msgr2=0x7f46080ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 shutdown_connections 2023-11-28T11:54:32.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.605+0000 7f460e2c5700 1 -- 172.21.15.144:0/2837648714 wait complete. 2023-11-28T11:54:32.629 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:54:32.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.747+0000 7fb7697da700 1 Processor -- start 2023-11-28T11:54:32.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7697da700 1 -- start start 2023-11-28T11:54:32.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7697da700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb76406fb30 0x7fb7640cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7697da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ce2e0 0x7fb76406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7697da700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76406bf10 con 0x7fb764158ce0 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7697da700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7640ce6f0 con 0x7fb76406fb30 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7697da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641590b0 con 0x7fb7640ce2e0 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb7627fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb76406fb30 0x7fb7640cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.748+0000 7fb762ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb764158ce0 0x7fb76415d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55542/0 (socket says 172.21.15.144:55542) 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb762ffd700 1 -- 172.21.15.144:0/3298443082 learned_addr learned my addr 172.21.15.144:0/3298443082 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:32.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7637fe700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ce2e0 0x7fb76406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7627fc700 1 -- 172.21.15.144:0/3298443082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ce2e0 msgr2=0x7fb76406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7627fc700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ce2e0 0x7fb76406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7627fc700 1 -- 172.21.15.144:0/3298443082 >> v1:172.21.15.144:6789/0 conn(0x7fb764158ce0 legacy=0x7fb76415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb761ffb700 1 -- 172.21.15.144:0/3298443082 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974193350 0 0) 0x7fb76406bf10 con 0x7fb764158ce0 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7627fc700 1 -- 172.21.15.144:0/3298443082 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7640cb0d0 con 0x7fb76406fb30 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7637fe700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ce2e0 0x7fb76406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:54:32.759 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb7627fc700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76406fb30 0x7fb7640cdda0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb75400bae0 tx=0x7fb75400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c900e4898f3eb77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb761ffb700 1 -- 172.21.15.144:0/3298443082 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75400dd30 con 0x7fb76406fb30 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb761ffb700 1 -- 172.21.15.144:0/3298443082 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75400de90 con 0x7fb76406fb30 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.749+0000 7fb761ffb700 1 -- 172.21.15.144:0/3298443082 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754010690 con 0x7fb76406fb30 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 -- 172.21.15.144:0/3298443082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76406fb30 msgr2=0x7fb7640cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76406fb30 0x7fb7640cdda0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb75400bae0 tx=0x7fb75400bdf0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 -- 172.21.15.144:0/3298443082 shutdown_connections 2023-11-28T11:54:32.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7640ce2e0 0x7fb76406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 --2- 172.21.15.144:0/3298443082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb76406fb30 0x7fb7640cdda0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 -- 172.21.15.144:0/3298443082 >> 172.21.15.144:0/3298443082 conn(0x7fb7640cb670 msgr2=0x7fb7640cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 -- 172.21.15.144:0/3298443082 shutdown_connections 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.750+0000 7fb7697da700 1 -- 172.21.15.144:0/3298443082 wait complete. 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.751+0000 7fb7697da700 1 Processor -- start 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.751+0000 7fb7697da700 1 -- start start 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7697da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb76406fb30 0x7fb7640c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7697da700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 0x7fb7641592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7697da700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb764158ce0 0x7fb764159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7697da700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76415bc20 con 0x7fb7640ce2e0 2023-11-28T11:54:32.761 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7697da700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76414d6d0 con 0x7fb764158ce0 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7697da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb764159ea0 con 0x7fb76406fb30 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 0x7fb7641592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 0x7fb7641592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38838/0 (socket says 172.21.15.144:38838) 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 -- 172.21.15.144:0/3113362614 learned_addr learned my addr 172.21.15.144:0/3113362614 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb762ffd700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb76406fb30 0x7fb7640c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7637fe700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb764158ce0 0x7fb764159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 -- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb76406fb30 msgr2=0x7fb7640c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb76406fb30 0x7fb7640c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 -- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb764158ce0 msgr2=0x7fb764159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb764158ce0 0x7fb764159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 -- 172.21.15.144:0/3113362614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75400b740 con 0x7fb7640ce2e0 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.752+0000 7fb7627fc700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 0x7fb7641592c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb754000c00 tx=0x7fb754004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.753+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75401f070 con 0x7fb7640ce2e0 2023-11-28T11:54:32.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.753+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754003b50 con 0x7fb7640ce2e0 2023-11-28T11:54:32.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.753+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76406b8c0 con 0x7fb7640ce2e0 2023-11-28T11:54:32.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.753+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754024470 con 0x7fb7640ce2e0 2023-11-28T11:54:32.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.753+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7640cb3b0 con 0x7fb7640ce2e0 2023-11-28T11:54:32.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.753+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76406bba0 con 0x7fb7640ce2e0 2023-11-28T11:54:32.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.754+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb754019400 con 0x7fb7640ce2e0 2023-11-28T11:54:32.764 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.754+0000 7fb74b7fe700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7500682f0 0x7fb75006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:32.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.754+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb754089360 con 0x7fb7640ce2e0 2023-11-28T11:54:32.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.754+0000 7fb762ffd700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7500682f0 0x7fb75006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:32.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.755+0000 7fb762ffd700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7500682f0 0x7fb75006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb758007fd0 tx=0x7fb758009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:32.765 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.759+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb754055ad0 con 0x7fb7640ce2e0 2023-11-28T11:54:32.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.984+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb7640c9670 con 0x7fb7640ce2e0 2023-11-28T11:54:32.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.984+0000 7fb74b7fe700 1 -- 172.21.15.144:0/3113362614 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb7540590f0 con 0x7fb7640ce2e0 2023-11-28T11:54:32.985 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:32.986 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:54:32.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7500682f0 msgr2=0x7fb75006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7500682f0 0x7fb75006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb758007fd0 tx=0x7fb758009040 comp rx=0 tx=0).stop 2023-11-28T11:54:32.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 msgr2=0x7fb7641592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:32.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 0x7fb7641592c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb754000c00 tx=0x7fb754004750 comp rx=0 tx=0).stop 2023-11-28T11:54:32.988 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 shutdown_connections 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fb7500682f0 0x7fb75006a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb764158ce0 0x7fb764159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb76406fb30 0x7fb7640c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 --2- 172.21.15.144:0/3113362614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7640ce2e0 0x7fb7641592c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 >> 172.21.15.144:0/3113362614 conn(0x7fb7640cb670 msgr2=0x7fb76406ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 shutdown_connections 2023-11-28T11:54:32.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:32.986+0000 7fb7697da700 1 -- 172.21.15.144:0/3113362614 wait complete. 2023-11-28T11:54:32.999 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:54:38.001 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:54:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.133+0000 7f8f2b3ea700 1 Processor -- start 2023-11-28T11:54:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.133+0000 7f8f2b3ea700 1 -- start start 2023-11-28T11:54:38.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f2b3ea700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8f2406ba90 0x7f8f240af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:38.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f2b3ea700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8f240cbdf0 0x7f8f240ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:38.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f2b3ea700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f240cc5f0 con 0x7f8f2415cef0 2023-11-28T11:54:38.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f2b3ea700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2415d470 con 0x7f8f240cbdf0 2023-11-28T11:54:38.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f2b3ea700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2415d750 con 0x7f8f2406ba90 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29987700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8f240cbdf0 0x7f8f240ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29186700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8f2415cef0 0x7f8f2406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55556/0 (socket says 172.21.15.144:55556) 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29186700 1 -- 172.21.15.144:0/4158191164 learned_addr learned my addr 172.21.15.144:0/4158191164 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f28985700 1 --2- 172.21.15.144:0/4158191164 >> v2:172.21.15.177:3300/0 conn(0x7f8f2406ba90 0x7f8f240af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29987700 1 -- 172.21.15.144:0/4158191164 >> v2:172.21.15.177:3300/0 conn(0x7f8f2406ba90 msgr2=0x7f8f240af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29987700 1 --2- 172.21.15.144:0/4158191164 >> v2:172.21.15.177:3300/0 conn(0x7f8f2406ba90 0x7f8f240af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29987700 1 -- 172.21.15.144:0/4158191164 >> v1:172.21.15.144:6789/0 conn(0x7f8f2415cef0 legacy=0x7f8f2406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f29987700 1 -- 172.21.15.144:0/4158191164 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2415da30 con 0x7f8f240cbdf0 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.134+0000 7f8f1bfff700 1 -- 172.21.15.144:0/4158191164 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959762211 0 0) 0x7f8f240cc5f0 con 0x7f8f2415cef0 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f29987700 1 --2- 172.21.15.144:0/4158191164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f240cbdf0 0x7f8f240ca650 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8f2406fbb0 tx=0x7f8f1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c67aa826238c212 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f1bfff700 1 -- 172.21.15.144:0/4158191164 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00e070 con 0x7f8f240cbdf0 2023-11-28T11:54:38.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f1bfff700 1 -- 172.21.15.144:0/4158191164 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c00bbd0 con 0x7f8f240cbdf0 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f1bfff700 1 -- 172.21.15.144:0/4158191164 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c012430 con 0x7f8f240cbdf0 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/4158191164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f240cbdf0 msgr2=0x7f8f240ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/4158191164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f240cbdf0 0x7f8f240ca650 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8f2406fbb0 tx=0x7f8f1c00b330 comp rx=0 tx=0).stop 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/4158191164 shutdown_connections 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/4158191164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f240cbdf0 0x7f8f240ca650 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/4158191164 >> v2:172.21.15.177:3300/0 conn(0x7f8f2406ba90 0x7f8f240af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/4158191164 >> 172.21.15.144:0/4158191164 conn(0x7f8f240d0000 msgr2=0x7f8f240d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/4158191164 shutdown_connections 2023-11-28T11:54:38.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.135+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/4158191164 wait complete. 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 Processor -- start 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 -- start start 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 0x7f8f24155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f240cbdf0 0x7f8f2406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f2415cef0 0x7f8f241532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24155a60 con 0x7f8f2406ba90 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24154020 con 0x7f8f2415cef0 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f2b3ea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24153d40 con 0x7f8f240cbdf0 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.137+0000 7f8f29186700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 0x7f8f24155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:38.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 0x7f8f24155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38850/0 (socket says 172.21.15.144:38850) 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 -- 172.21.15.144:0/2577708292 learned_addr learned my addr 172.21.15.144:0/2577708292 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29987700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f2415cef0 0x7f8f241532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f28985700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f240cbdf0 0x7f8f2406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 -- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f240cbdf0 msgr2=0x7f8f2406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f240cbdf0 0x7f8f2406aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 -- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f2415cef0 msgr2=0x7f8f241532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f2415cef0 0x7f8f241532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 -- 172.21.15.144:0/2577708292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c009a70 con 0x7f8f2406ba90 2023-11-28T11:54:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f29186700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 0x7f8f24155ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c550 tx=0x7f8f1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14004c80 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14004e00 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14007310 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2415dff0 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.138+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2415dd10 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.139+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f240cab90 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.139+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8f1401a020 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.140+0000 7f8f19ffb700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8f080682f0 0x7f8f0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.140+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8f140858d0 con 0x7f8f2406ba90 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.140+0000 7f8f28985700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8f080682f0 0x7f8f0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:38.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.140+0000 7f8f28985700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8f080682f0 0x7f8f0806a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8f2415cc50 tx=0x7f8f20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.144+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8f14052150 con 0x7f8f2406ba90 2023-11-28T11:54:38.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.367+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8f240c38b0 con 0x7f8f2406ba90 2023-11-28T11:54:38.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.368+0000 7f8f19ffb700 1 -- 172.21.15.144:0/2577708292 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8f14055770 con 0x7f8f2406ba90 2023-11-28T11:54:38.369 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:38.369 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8f080682f0 msgr2=0x7f8f0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8f080682f0 0x7f8f0806a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8f2415cc50 tx=0x7f8f20008040 comp rx=0 tx=0).stop 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 msgr2=0x7f8f24155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 0x7f8f24155ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c550 tx=0x7f8f1400c910 comp rx=0 tx=0).stop 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 shutdown_connections 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f2406ba90 0x7f8f24155ef0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8f080682f0 0x7f8f0806a7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f2415cef0 0x7f8f241532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 --2- 172.21.15.144:0/2577708292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f240cbdf0 0x7f8f2406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 >> 172.21.15.144:0/2577708292 conn(0x7f8f240d0000 msgr2=0x7f8f2406dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:38.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 shutdown_connections 2023-11-28T11:54:38.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:38.370+0000 7f8f2b3ea700 1 -- 172.21.15.144:0/2577708292 wait complete. 2023-11-28T11:54:38.382 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:54:43.385 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:54:43.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.516+0000 7f5c42486700 1 Processor -- start 2023-11-28T11:54:43.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.516+0000 7f5c42486700 1 -- start start 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c42486700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5c3c140950 0x7f5c3c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c42486700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c0b2e70 0x7f5c3c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c42486700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3c13ff00 con 0x7f5c3c0cd8c0 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c42486700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3c1404c0 con 0x7f5c3c140950 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c42486700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c140dc0 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5c3c140950 0x7f5c3c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c40a23700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c0b2e70 0x7f5c3c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3bfff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5c3c0cd8c0 0x7f5c3c0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:56060/0 (socket says 172.21.15.144:56060) 2023-11-28T11:54:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3bfff700 1 -- 172.21.15.144:0/3780749142 learned_addr learned my addr 172.21.15.144:0/3780749142 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3affd700 1 -- 172.21.15.144:0/3780749142 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610706287 0 0) 0x7f5c3c13ff00 con 0x7f5c3c0cd8c0 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3affd700 1 -- 172.21.15.144:0/3780749142 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c240035b0 con 0x7f5c3c0cd8c0 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3b7fe700 1 -- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c0b2e70 msgr2=0x7f5c3c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3b7fe700 1 --2- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c0b2e70 0x7f5c3c06f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.517+0000 7f5c3b7fe700 1 -- 172.21.15.144:0/3780749142 >> v1:172.21.15.177:6789/0 conn(0x7f5c3c0cd8c0 legacy=0x7f5c3c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c3b7fe700 1 -- 172.21.15.144:0/3780749142 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c15aa80 con 0x7f5c3c140950 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c40a23700 1 --2- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c0b2e70 0x7f5c3c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c3b7fe700 1 --2- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c140950 0x7f5c3c0b2930 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009840 tx=0x7f5c2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5502f85917f5431 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c3affd700 1 -- 172.21.15.144:0/3780749142 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c011830 con 0x7f5c3c140950 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c3affd700 1 -- 172.21.15.144:0/3780749142 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c011990 con 0x7f5c3c140950 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c3affd700 1 -- 172.21.15.144:0/3780749142 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c011d00 con 0x7f5c3c140950 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c42486700 1 -- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c140950 msgr2=0x7f5c3c0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c42486700 1 --2- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c140950 0x7f5c3c0b2930 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009840 tx=0x7f5c2c009b50 comp rx=0 tx=0).stop 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c42486700 1 -- 172.21.15.144:0/3780749142 shutdown_connections 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c42486700 1 --2- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c0b2e70 0x7f5c3c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c42486700 1 --2- 172.21.15.144:0/3780749142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c140950 0x7f5c3c0b2930 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.518+0000 7f5c42486700 1 -- 172.21.15.144:0/3780749142 >> 172.21.15.144:0/3780749142 conn(0x7f5c3c06aed0 msgr2=0x7f5c3c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.519+0000 7f5c42486700 1 -- 172.21.15.144:0/3780749142 shutdown_connections 2023-11-28T11:54:43.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.519+0000 7f5c42486700 1 -- 172.21.15.144:0/3780749142 wait complete. 2023-11-28T11:54:43.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.520+0000 7f5c42486700 1 Processor -- start 2023-11-28T11:54:43.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.520+0000 7f5c42486700 1 -- start start 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c42486700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 0x7f5c3c1f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c42486700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c0cd8c0 0x7f5c3c142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c42486700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c140950 0x7f5c3c1f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c42486700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c153a80 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c42486700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c13db40 con 0x7f5c3c0cd8c0 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c42486700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c13dd90 con 0x7f5c3c140950 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 0x7f5c3c1f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 0x7f5c3c1f13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38864/0 (socket says 172.21.15.144:38864) 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 -- 172.21.15.144:0/3156008937 learned_addr learned my addr 172.21.15.144:0/3156008937 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3b7fe700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c0cd8c0 0x7f5c3c142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c40a23700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c140950 0x7f5c3c1f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 -- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c140950 msgr2=0x7f5c3c1f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c140950 0x7f5c3c1f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 -- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c0cd8c0 msgr2=0x7f5c3c142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c0cd8c0 0x7f5c3c142ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 -- 172.21.15.144:0/3156008937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009a70 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3b7fe700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c0cd8c0 0x7f5c3c142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c3bfff700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 0x7f5c3c1f13c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c4d0 tx=0x7f5c3000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005dc0 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.521+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c300070d0 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.522+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30007480 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.522+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c0094a0 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.522+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c0b35f0 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.522+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c0b3310 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.523+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c30019030 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.523+0000 7f5c38ff9700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c280682f0 0x7f5c2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.523+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5c30085150 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.523+0000 7f5c3b7fe700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c280682f0 0x7f5c2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.524+0000 7f5c3b7fe700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c280682f0 0x7f5c2806a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009840 tx=0x7f5c2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:43.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.528+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c30051920 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.753+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5c3c140150 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.753+0000 7f5c38ff9700 1 -- 172.21.15.144:0/3156008937 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5c30054f40 con 0x7f5c3c0b2e70 2023-11-28T11:54:43.754 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:43.754 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:54:43.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c280682f0 msgr2=0x7f5c2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:43.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c280682f0 0x7f5c2806a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009840 tx=0x7f5c2c008030 comp rx=0 tx=0).stop 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 msgr2=0x7f5c3c1f13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 0x7f5c3c1f13c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000c4d0 tx=0x7f5c3000c890 comp rx=0 tx=0).stop 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 shutdown_connections 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0b2e70 0x7f5c3c1f13c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5c280682f0 0x7f5c2806a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5c3c140950 0x7f5c3c1f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 --2- 172.21.15.144:0/3156008937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3c0cd8c0 0x7f5c3c142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:43.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.755+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 >> 172.21.15.144:0/3156008937 conn(0x7f5c3c06aed0 msgr2=0x7f5c3c0cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:43.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.756+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 shutdown_connections 2023-11-28T11:54:43.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:43.756+0000 7f5c42486700 1 -- 172.21.15.144:0/3156008937 wait complete. 2023-11-28T11:54:43.768 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:54:48.769 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:54:48.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.899+0000 7f911cd85700 1 Processor -- start 2023-11-28T11:54:48.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.900+0000 7f911cd85700 1 -- start start 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911cd85700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9118071040 0x7f9118071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911cd85700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 0x7f91181609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911cd85700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9118071950 con 0x7f911806dee0 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911cd85700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911815d060 con 0x7f9118071040 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911cd85700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911815d340 con 0x7f9118160540 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f9115d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f911806dee0 0x7f911806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55584/0 (socket says 172.21.15.144:55584) 2023-11-28T11:54:48.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f9115d9b700 1 -- 172.21.15.144:0/596879103 learned_addr learned my addr 172.21.15.144:0/596879103 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:48.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f9116d9d700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 0x7f91181609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816301370 0 0) 0x7f9118071950 con 0x7f911806dee0 2023-11-28T11:54:48.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911659c700 1 --2- 172.21.15.144:0/596879103 >> v2:172.21.15.177:3300/0 conn(0x7f9118071040 0x7f9118071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91000035b0 con 0x7f911806dee0 2023-11-28T11:54:48.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569370486 0 0) 0x7f91000035b0 con 0x7f911806dee0 2023-11-28T11:54:48.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 msgr2=0x7f91181609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 0x7f91181609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 >> v2:172.21.15.177:3300/0 conn(0x7f9118071040 msgr2=0x7f9118071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 --2- 172.21.15.144:0/596879103 >> v2:172.21.15.177:3300/0 conn(0x7f9118071040 0x7f9118071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911815d620 con 0x7f911806dee0 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f9108001670 con 0x7f911806dee0 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 >> v1:172.21.15.144:6789/0 conn(0x7f911806dee0 legacy=0x7f911806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9100003a00 0x7f9100003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9100004850 0x7f9100004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 0x7f9100005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911815d060 con 0x7f9100003a00 2023-11-28T11:54:48.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911815d340 con 0x7f9100004850 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910000ba90 con 0x7f9118160540 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.901+0000 7f911659c700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9100003a00 0x7f9100003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f9115d9b700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9100004850 0x7f9100004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f9116d9d700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 0x7f9100005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911659c700 1 -- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 msgr2=0x7f9100005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911659c700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9118160540 0x7f9100005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911659c700 1 -- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9100004850 msgr2=0x7f9100004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911659c700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9100004850 0x7f9100004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911659c700 1 -- 172.21.15.144:0/596879103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000035b0 con 0x7f9100003a00 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911659c700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9100003a00 0x7f9100003e10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f910c009b60 tx=0x7f9100005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4f58e5b56dc120 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:48.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00e6c0 con 0x7f9100003a00 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c00e820 con 0x7f9100003a00 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.902+0000 7f911559a700 1 -- 172.21.15.144:0/596879103 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00eb90 con 0x7f9100003a00 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f911cd85700 1 -- 172.21.15.144:0/596879103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9100003a00 msgr2=0x7f9100003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f911cd85700 1 --2- 172.21.15.144:0/596879103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9100003a00 0x7f9100003e10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f910c009b60 tx=0x7f9100005880 comp rx=0 tx=0).stop 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f9116d9d700 1 -- 172.21.15.144:0/596879103 reap_dead start 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f911cd85700 1 -- 172.21.15.144:0/596879103 shutdown_connections 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f911cd85700 1 -- 172.21.15.144:0/596879103 >> 172.21.15.144:0/596879103 conn(0x7f91180d0000 msgr2=0x7f91180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f911cd85700 1 -- 172.21.15.144:0/596879103 shutdown_connections 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.903+0000 7f911cd85700 1 -- 172.21.15.144:0/596879103 wait complete. 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.904+0000 7f911cd85700 1 Processor -- start 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.904+0000 7f911cd85700 1 -- start start 2023-11-28T11:54:48.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911cd85700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100003a00 0x7f91181f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911cd85700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118155950 0x7f91181537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911cd85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 0x7f91180cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911cd85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181f69c0 con 0x7f91180cf090 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911cd85700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180ae4f0 con 0x7f9118155950 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911cd85700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180cfa00 con 0x7f9100003a00 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 0x7f91180cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 0x7f91180cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38886/0 (socket says 172.21.15.144:38886) 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 -- 172.21.15.144:0/811199501 learned_addr learned my addr 172.21.15.144:0/811199501 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f911659c700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100003a00 0x7f91181f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9115d9b700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118155950 0x7f91181537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 -- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100003a00 msgr2=0x7f91181f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100003a00 0x7f91181f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 -- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118155950 msgr2=0x7f91181537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118155950 0x7f91181537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 -- 172.21.15.144:0/811199501 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911815ccd0 con 0x7f91180cf090 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.905+0000 7f9116d9d700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 0x7f91180cf4c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9118071d40 tx=0x7f9110011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.906+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000e040 con 0x7f91180cf090 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.906+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911000f040 con 0x7f91180cf090 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.906+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911815d900 con 0x7f91180cf090 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.906+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000fe40 con 0x7f91180cf090 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.906+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911806a270 con 0x7f91180cf090 2023-11-28T11:54:48.915 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.907+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91180ae680 con 0x7f91180cf090 2023-11-28T11:54:48.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.907+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9110015430 con 0x7f91180cf090 2023-11-28T11:54:48.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.907+0000 7f9106ffd700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f90f80682f0 0x7f90f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:48.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.908+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9110086280 con 0x7f91180cf090 2023-11-28T11:54:48.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.908+0000 7f911659c700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f90f80682f0 0x7f90f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:48.916 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.908+0000 7f911659c700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f90f80682f0 0x7f90f806a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f910c0090b0 tx=0x7f910c01f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:48.917 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:48.912+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91100529f0 con 0x7f91180cf090 2023-11-28T11:54:49.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.137+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f91180720a0 con 0x7f91180cf090 2023-11-28T11:54:49.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.137+0000 7f9106ffd700 1 -- 172.21.15.144:0/811199501 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9110056010 con 0x7f91180cf090 2023-11-28T11:54:49.139 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:49.139 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f90f80682f0 msgr2=0x7f90f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f90f80682f0 0x7f90f806a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f910c0090b0 tx=0x7f910c01f040 comp rx=0 tx=0).stop 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 msgr2=0x7f91180cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 0x7f91180cf4c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9118071d40 tx=0x7f9110011f60 comp rx=0 tx=0).stop 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 shutdown_connections 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91180cf090 0x7f91180cf4c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f90f80682f0 0x7f90f806a7a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9100003a00 0x7f91181f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:49.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 --2- 172.21.15.144:0/811199501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9118155950 0x7f91181537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:49.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 >> 172.21.15.144:0/811199501 conn(0x7f91180d0000 msgr2=0x7f911806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:49.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 shutdown_connections 2023-11-28T11:54:49.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:49.139+0000 7f911cd85700 1 -- 172.21.15.144:0/811199501 wait complete. 2023-11-28T11:54:49.153 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:54:53.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:54:53.150 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:54:54.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:54:54.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.283+0000 7f9a98ea9700 1 Processor -- start 2023-11-28T11:54:54.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.283+0000 7f9a98ea9700 1 -- start start 2023-11-28T11:54:54.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a98ea9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cd8c0 0x7f9a94140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:54.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a98ea9700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a940cdc90 0x7f9a940b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:54.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a98ea9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9413ff00 con 0x7f9a940b2e70 2023-11-28T11:54:54.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a98ea9700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a941404c0 con 0x7f9a940cdc90 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a98ea9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9415aa80 con 0x7f9a940cd8c0 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a91d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a940cdc90 0x7f9a940b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a92d9d700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9a940b2e70 0x7f9a9406f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55604/0 (socket says 172.21.15.144:55604) 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a91d9b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a940cdc90 0x7f9a940b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38890/0 (socket says 172.21.15.144:38890) 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a92d9d700 1 -- 172.21.15.144:0/1394404460 learned_addr learned my addr 172.21.15.144:0/1394404460 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a9259c700 1 --2- 172.21.15.144:0/1394404460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cd8c0 0x7f9a94140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a91d9b700 1 -- 172.21.15.144:0/1394404460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cd8c0 msgr2=0x7f9a94140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a91d9b700 1 --2- 172.21.15.144:0/1394404460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cd8c0 0x7f9a94140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a91d9b700 1 -- 172.21.15.144:0/1394404460 >> v1:172.21.15.144:6789/0 conn(0x7f9a940b2e70 legacy=0x7f9a9406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:54.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a9159a700 1 -- 172.21.15.144:0/1394404460 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755778458 0 0) 0x7f9a9413ff00 con 0x7f9a940b2e70 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.284+0000 7f9a91d9b700 1 -- 172.21.15.144:0/1394404460 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940b3310 con 0x7f9a940cdc90 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a91d9b700 1 --2- 172.21.15.144:0/1394404460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940cdc90 0x7f9a940b2930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009840 tx=0x7f9a88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c459b802ab1bd21e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a9159a700 1 -- 172.21.15.144:0/1394404460 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88011830 con 0x7f9a940cdc90 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a9159a700 1 -- 172.21.15.144:0/1394404460 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88011990 con 0x7f9a940cdc90 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a9159a700 1 -- 172.21.15.144:0/1394404460 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88011d20 con 0x7f9a940cdc90 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1394404460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940cdc90 msgr2=0x7f9a940b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1394404460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940cdc90 0x7f9a940b2930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009840 tx=0x7f9a88009b50 comp rx=0 tx=0).stop 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1394404460 shutdown_connections 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1394404460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cd8c0 0x7f9a94140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1394404460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940cdc90 0x7f9a940b2930 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1394404460 >> 172.21.15.144:0/1394404460 conn(0x7f9a9406aed0 msgr2=0x7f9a9406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:54.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1394404460 shutdown_connections 2023-11-28T11:54:54.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.285+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1394404460 wait complete. 2023-11-28T11:54:54.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.287+0000 7f9a98ea9700 1 Processor -- start 2023-11-28T11:54:54.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.287+0000 7f9a98ea9700 1 -- start start 2023-11-28T11:54:54.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.287+0000 7f9a98ea9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 0x7f9a9413e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.287+0000 7f9a98ea9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a940cd8c0 0x7f9a940b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.287+0000 7f9a98ea9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cdc90 0x7f9a940b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a98ea9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9413f6d0 con 0x7f9a940b2e70 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a98ea9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9415aed0 con 0x7f9a940cd8c0 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a98ea9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9413ee30 con 0x7f9a940cdc90 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 0x7f9a9413e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 0x7f9a9413e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38894/0 (socket says 172.21.15.144:38894) 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 -- 172.21.15.144:0/1578098526 learned_addr learned my addr 172.21.15.144:0/1578098526 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:54.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 -- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cdc90 msgr2=0x7f9a940b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cdc90 0x7f9a940b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a91d9b700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a940cd8c0 0x7f9a940b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 -- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a940cd8c0 msgr2=0x7f9a940b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a940cd8c0 0x7f9a940b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 -- 172.21.15.144:0/1578098526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880094a0 con 0x7f9a940b2e70 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a9259c700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 0x7f9a9413e520 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400e9b0 tx=0x7f9a8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84009710 con 0x7f9a940b2e70 2023-11-28T11:54:54.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8400c850 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a94140150 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400cbe0 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.288+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a940b35f0 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.289+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a940ce100 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.290+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a84023070 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.290+0000 7f9a82ffd700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9a740682f0 0x7f9a7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.290+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9a84086050 con 0x7f9a940b2e70 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.290+0000 7f9a91d9b700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9a740682f0 0x7f9a7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:54.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.291+0000 7f9a91d9b700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9a740682f0 0x7f9a7406a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009840 tx=0x7f9a88008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:54.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.294+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a840527c0 con 0x7f9a940b2e70 2023-11-28T11:54:54.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.518+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9a94068690 con 0x7f9a940b2e70 2023-11-28T11:54:54.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.519+0000 7f9a82ffd700 1 -- 172.21.15.144:0/1578098526 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9a84055de0 con 0x7f9a940b2e70 2023-11-28T11:54:54.521 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:54.521 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9a740682f0 msgr2=0x7f9a7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9a740682f0 0x7f9a7406a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009840 tx=0x7f9a88008030 comp rx=0 tx=0).stop 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 msgr2=0x7f9a9413e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 0x7f9a9413e520 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400e9b0 tx=0x7f9a8400ed70 comp rx=0 tx=0).stop 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 shutdown_connections 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a940b2e70 0x7f9a9413e520 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9a740682f0 0x7f9a7406a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a940cdc90 0x7f9a940b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 --2- 172.21.15.144:0/1578098526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a940cd8c0 0x7f9a940b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:54.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 >> 172.21.15.144:0/1578098526 conn(0x7f9a9406aed0 msgr2=0x7f9a940b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:54.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 shutdown_connections 2023-11-28T11:54:54.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:54.521+0000 7f9a98ea9700 1 -- 172.21.15.144:0/1578098526 wait complete. 2023-11-28T11:54:54.535 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:54:59.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:54:59.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.667+0000 7f88f05d6700 1 Processor -- start 2023-11-28T11:54:59.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.667+0000 7f88f05d6700 1 -- start start 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88f05d6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f88e815c9b0 0x7f88e80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88f05d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e80cd7c0 0x7f88e80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88f05d6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80d1950 con 0x7f88e8161340 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88f05d6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e80b5800 con 0x7f88e815c9b0 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88f05d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80ce140 con 0x7f88e80cd7c0 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f88e815c9b0 0x7f88e80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:59.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f88e815c9b0 0x7f88e80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38902/0 (socket says 172.21.15.144:38902) 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 -- 172.21.15.144:0/1756116699 learned_addr learned my addr 172.21.15.144:0/1756116699 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88eeb73700 1 --2- 172.21.15.144:0/1756116699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e80cd7c0 0x7f88e80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 -- 172.21.15.144:0/1756116699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e80cd7c0 msgr2=0x7f88e80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 --2- 172.21.15.144:0/1756116699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e80cd7c0 0x7f88e80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 -- 172.21.15.144:0/1756116699 >> v1:172.21.15.177:6789/0 conn(0x7f88e8161340 legacy=0x7f88e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 -- 172.21.15.144:0/1756116699 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80d24f0 con 0x7f88e815c9b0 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.668+0000 7f88ee372700 1 --2- 172.21.15.144:0/1756116699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e80b4bf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098f0 tx=0x7f88e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe3f42d5dda4d34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88ed370700 1 -- 172.21.15.144:0/1756116699 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e400b8a0 con 0x7f88e815c9b0 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88ed370700 1 -- 172.21.15.144:0/1756116699 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e400ba00 con 0x7f88e815c9b0 2023-11-28T11:54:59.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88ed370700 1 -- 172.21.15.144:0/1756116699 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e400bd90 con 0x7f88e815c9b0 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 -- 172.21.15.144:0/1756116699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 msgr2=0x7f88e80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 --2- 172.21.15.144:0/1756116699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e80b4bf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098f0 tx=0x7f88e4009c00 comp rx=0 tx=0).stop 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 -- 172.21.15.144:0/1756116699 shutdown_connections 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 --2- 172.21.15.144:0/1756116699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e80cd7c0 0x7f88e80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 --2- 172.21.15.144:0/1756116699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e80b4bf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 -- 172.21.15.144:0/1756116699 >> 172.21.15.144:0/1756116699 conn(0x7f88e8072600 msgr2=0x7f88e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 -- 172.21.15.144:0/1756116699 shutdown_connections 2023-11-28T11:54:59.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.669+0000 7f88f05d6700 1 -- 172.21.15.144:0/1756116699 wait complete. 2023-11-28T11:54:59.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.670+0000 7f88f05d6700 1 Processor -- start 2023-11-28T11:54:59.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 -- start start 2023-11-28T11:54:59.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80cd7c0 0x7f88e8147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:59.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e81e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:59.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e8161340 0x7f88e8159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:59.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80c6780 con 0x7f88e815c9b0 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8147970 con 0x7f88e80cd7c0 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88f05d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81e6d60 con 0x7f88e8161340 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88edb71700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e81e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88edb71700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e81e6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38910/0 (socket says 172.21.15.144:38910) 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88edb71700 1 -- 172.21.15.144:0/3444391907 learned_addr learned my addr 172.21.15.144:0/3444391907 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88eeb73700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e8161340 0x7f88e8159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.671+0000 7f88ee372700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80cd7c0 0x7f88e8147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88edb71700 1 -- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e8161340 msgr2=0x7f88e8159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88edb71700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e8161340 0x7f88e8159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88edb71700 1 -- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80cd7c0 msgr2=0x7f88e8147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:59.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88edb71700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80cd7c0 0x7f88e8147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88edb71700 1 -- 172.21.15.144:0/3444391907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4009550 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88edb71700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e81e6480 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c4e0 tx=0x7f88d800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80052d0 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8005430 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e80d1300 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80038c0 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.672+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e80b5520 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.673+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e80d15e0 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.673+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88d8019030 con 0x7f88e815c9b0 2023-11-28T11:54:59.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.674+0000 7f88deffd700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f88cc0682f0 0x7f88cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:54:59.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.674+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f88d8084f80 con 0x7f88e815c9b0 2023-11-28T11:54:59.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.674+0000 7f88ee372700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f88cc0682f0 0x7f88cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:54:59.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.674+0000 7f88ee372700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f88cc0682f0 0x7f88cc06a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f88e4014040 tx=0x7f88e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:54:59.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.678+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88d8051750 con 0x7f88e815c9b0 2023-11-28T11:54:59.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.902+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f88e80745c0 con 0x7f88e815c9b0 2023-11-28T11:54:59.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.902+0000 7f88deffd700 1 -- 172.21.15.144:0/3444391907 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f88d80170c0 con 0x7f88e815c9b0 2023-11-28T11:54:59.904 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:54:59.904 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:54:59.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f88cc0682f0 msgr2=0x7f88cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:59.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f88cc0682f0 0x7f88cc06a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f88e4014040 tx=0x7f88e4008030 comp rx=0 tx=0).stop 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 msgr2=0x7f88e81e6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e81e6480 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f88d800c4e0 tx=0x7f88d800c7f0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 shutdown_connections 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f88cc0682f0 0x7f88cc06a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88e8161340 0x7f88e8159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88e80cd7c0 0x7f88e8147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 --2- 172.21.15.144:0/3444391907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88e815c9b0 0x7f88e81e6480 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.904+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 >> 172.21.15.144:0/3444391907 conn(0x7f88e8072600 msgr2=0x7f88e80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.905+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 shutdown_connections 2023-11-28T11:54:59.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:54:59.905+0000 7f88f05d6700 1 -- 172.21.15.144:0/3444391907 wait complete. 2023-11-28T11:54:59.918 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:04.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:05.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.051+0000 7f9515a93700 1 Processor -- start 2023-11-28T11:55:05.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.051+0000 7f9515a93700 1 -- start start 2023-11-28T11:55:05.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9515a93700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f95100cd8c0 0x7f9510140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9515a93700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95100cdc90 0x7f95100b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9515a93700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100b2e70 0x7f951006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9515a93700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951013ff00 con 0x7f95100cdc90 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9515a93700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95101404c0 con 0x7f95100cd8c0 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9515a93700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951015aa80 con 0x7f95100b2e70 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95100cdc90 0x7f95100b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95100cdc90 0x7f95100b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38918/0 (socket says 172.21.15.144:38918) 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 -- 172.21.15.144:0/813101959 learned_addr learned my addr 172.21.15.144:0/813101959 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f950ffff700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100b2e70 0x7f951006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f950f7fe700 1 --2- 172.21.15.144:0/813101959 >> v2:172.21.15.177:3300/0 conn(0x7f95100cd8c0 0x7f9510140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 -- 172.21.15.144:0/813101959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100b2e70 msgr2=0x7f951006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100b2e70 0x7f951006f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 -- 172.21.15.144:0/813101959 >> v2:172.21.15.177:3300/0 conn(0x7f95100cd8c0 msgr2=0x7f9510140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 --2- 172.21.15.144:0/813101959 >> v2:172.21.15.177:3300/0 conn(0x7f95100cd8c0 0x7f9510140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 -- 172.21.15.144:0/813101959 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100b3310 con 0x7f95100cdc90 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f950f7fe700 1 --2- 172.21.15.144:0/813101959 >> v2:172.21.15.177:3300/0 conn(0x7f95100cd8c0 0x7f9510140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f950ffff700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100b2e70 0x7f951006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.052+0000 7f9506ffd700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100cdc90 0x7f95100b2930 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9510070a20 tx=0x7f9500009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a8c9b37a038957 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f950effd700 1 -- 172.21.15.144:0/813101959 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000b870 con 0x7f95100cdc90 2023-11-28T11:55:05.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f950effd700 1 -- 172.21.15.144:0/813101959 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950000b9d0 con 0x7f95100cdc90 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f950effd700 1 -- 172.21.15.144:0/813101959 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000bd60 con 0x7f95100cdc90 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 -- 172.21.15.144:0/813101959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100cdc90 msgr2=0x7f95100b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100cdc90 0x7f95100b2930 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9510070a20 tx=0x7f9500009bd0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 -- 172.21.15.144:0/813101959 shutdown_connections 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100b2e70 0x7f951006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 --2- 172.21.15.144:0/813101959 >> v2:172.21.15.177:3300/0 conn(0x7f95100cd8c0 0x7f9510140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 --2- 172.21.15.144:0/813101959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100cdc90 0x7f95100b2930 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 -- 172.21.15.144:0/813101959 >> 172.21.15.144:0/813101959 conn(0x7f951006aed0 msgr2=0x7f951006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 -- 172.21.15.144:0/813101959 shutdown_connections 2023-11-28T11:55:05.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.053+0000 7f9515a93700 1 -- 172.21.15.144:0/813101959 wait complete. 2023-11-28T11:55:05.061 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.054+0000 7f9515a93700 1 Processor -- start 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 -- start start 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 0x7f9510154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95100cd8c0 0x7f9510142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100cdc90 0x7f95101e4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95101549d0 con 0x7f95100b2e70 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9510141ce0 con 0x7f95100cd8c0 2023-11-28T11:55:05.062 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f9515a93700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9510141fc0 con 0x7f95100cdc90 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f950f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 0x7f9510154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f950f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 0x7f9510154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38922/0 (socket says 172.21.15.144:38922) 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.055+0000 7f950f7fe700 1 -- 172.21.15.144:0/2472944411 learned_addr learned my addr 172.21.15.144:0/2472944411 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950ffff700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100cdc90 0x7f95101e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f9506ffd700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95100cd8c0 0x7f9510142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950f7fe700 1 -- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100cdc90 msgr2=0x7f95101e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950f7fe700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100cdc90 0x7f95101e4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950f7fe700 1 -- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95100cd8c0 msgr2=0x7f9510142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950f7fe700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95100cd8c0 0x7f9510142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950f7fe700 1 -- 172.21.15.144:0/2472944411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9500009550 con 0x7f95100b2e70 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950f7fe700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 0x7f9510154490 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e7f0 tx=0x7f94f800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:05.063 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8005c20 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f8005d80 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9510140150 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8006270 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.056+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95100b35f0 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.057+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95100ce100 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.058+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94f8022070 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.058+0000 7f950cff9700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94f00682f0 0x7f94f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.058+0000 7f9506ffd700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94f00682f0 0x7f94f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.058+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f94f8085650 con 0x7f95100b2e70 2023-11-28T11:55:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.058+0000 7f9506ffd700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94f00682f0 0x7f94f006a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9510070a20 tx=0x7f9500008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:05.065 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.062+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94f8051e20 con 0x7f95100b2e70 2023-11-28T11:55:05.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.284+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9510068690 con 0x7f95100b2e70 2023-11-28T11:55:05.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.284+0000 7f950cff9700 1 -- 172.21.15.144:0/2472944411 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f94f8009570 con 0x7f95100b2e70 2023-11-28T11:55:05.286 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:05.286 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:05.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94f00682f0 msgr2=0x7f94f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:05.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94f00682f0 0x7f94f006a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9510070a20 tx=0x7f9500008030 comp rx=0 tx=0).stop 2023-11-28T11:55:05.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 msgr2=0x7f9510154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:05.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 0x7f9510154490 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f94f800e7f0 tx=0x7f94f800eb00 comp rx=0 tx=0).stop 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 shutdown_connections 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95100b2e70 0x7f9510154490 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f94f00682f0 0x7f94f006a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95100cdc90 0x7f95101e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 --2- 172.21.15.144:0/2472944411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95100cd8c0 0x7f9510142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 >> 172.21.15.144:0/2472944411 conn(0x7f951006aed0 msgr2=0x7f95100cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 shutdown_connections 2023-11-28T11:55:05.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:05.286+0000 7f9515a93700 1 -- 172.21.15.144:0/2472944411 wait complete. 2023-11-28T11:55:05.300 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:10.301 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:10.432 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.432+0000 7f7a4c54f700 1 Processor -- start 2023-11-28T11:55:10.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.432+0000 7f7a4c54f700 1 -- start start 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4c54f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a44071040 0x7f7a44071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4c54f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7a44160540 0x7f7a44160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4c54f700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a44071950 con 0x7f7a4406dee0 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4c54f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4415d060 con 0x7f7a44071040 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4c54f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4415d340 con 0x7f7a44160540 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4a2eb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a44071040 0x7f7a44071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a49aea700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7a4406dee0 0x7f7a4406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55642/0 (socket says 172.21.15.144:55642) 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a49aea700 1 -- 172.21.15.144:0/746895870 learned_addr learned my addr 172.21.15.144:0/746895870 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:55:10.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4aaec700 1 --2- 172.21.15.144:0/746895870 >> v2:172.21.15.177:3300/0 conn(0x7f7a44160540 0x7f7a44160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4a2eb700 1 -- 172.21.15.144:0/746895870 >> v2:172.21.15.177:3300/0 conn(0x7f7a44160540 msgr2=0x7f7a44160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4a2eb700 1 --2- 172.21.15.144:0/746895870 >> v2:172.21.15.177:3300/0 conn(0x7f7a44160540 0x7f7a44160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4a2eb700 1 -- 172.21.15.144:0/746895870 >> v1:172.21.15.144:6789/0 conn(0x7f7a4406dee0 legacy=0x7f7a4406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a492e9700 1 -- 172.21.15.144:0/746895870 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546544418 0 0) 0x7f7a44071950 con 0x7f7a4406dee0 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4a2eb700 1 -- 172.21.15.144:0/746895870 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4415d620 con 0x7f7a44071040 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a4a2eb700 1 --2- 172.21.15.144:0/746895870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a44071410 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7a440c3a10 tx=0x7f7a40009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dffcf9bb32f47ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a492e9700 1 -- 172.21.15.144:0/746895870 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40011830 con 0x7f7a44071040 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a492e9700 1 -- 172.21.15.144:0/746895870 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40011990 con 0x7f7a44071040 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.433+0000 7f7a492e9700 1 -- 172.21.15.144:0/746895870 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40011d20 con 0x7f7a44071040 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 -- 172.21.15.144:0/746895870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 msgr2=0x7f7a44071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/746895870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a44071410 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7a440c3a10 tx=0x7f7a40009b20 comp rx=0 tx=0).stop 2023-11-28T11:55:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 -- 172.21.15.144:0/746895870 shutdown_connections 2023-11-28T11:55:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/746895870 >> v2:172.21.15.177:3300/0 conn(0x7f7a44160540 0x7f7a44160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/746895870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a44071410 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 -- 172.21.15.144:0/746895870 >> 172.21.15.144:0/746895870 conn(0x7f7a440d0000 msgr2=0x7f7a440d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 -- 172.21.15.144:0/746895870 shutdown_connections 2023-11-28T11:55:10.437 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.434+0000 7f7a4c54f700 1 -- 172.21.15.144:0/746895870 wait complete. 2023-11-28T11:55:10.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.435+0000 7f7a4c54f700 1 Processor -- start 2023-11-28T11:55:10.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.435+0000 7f7a4c54f700 1 -- start start 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4c54f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4406dee0 0x7f7a441f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4c54f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a440b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4c54f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a44160540 0x7f7a440cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4c54f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c68b0 con 0x7f7a44071040 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4c54f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c3b40 con 0x7f7a4406dee0 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4c54f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441f60b0 con 0x7f7a44160540 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a49aea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a440b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a49aea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a440b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38936/0 (socket says 172.21.15.144:38936) 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a49aea700 1 -- 172.21.15.144:0/1545035129 learned_addr learned my addr 172.21.15.144:0/1545035129 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4aaec700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a44160540 0x7f7a440cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:10.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a4a2eb700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4406dee0 0x7f7a441f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a49aea700 1 -- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a44160540 msgr2=0x7f7a440cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a49aea700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a44160540 0x7f7a440cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.436+0000 7f7a49aea700 1 -- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4406dee0 msgr2=0x7f7a441f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a49aea700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4406dee0 0x7f7a441f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a49aea700 1 -- 172.21.15.144:0/1545035129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400094a0 con 0x7f7a44071040 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a49aea700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a440b0e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400c580 tx=0x7f7a3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34004c80 con 0x7f7a44071040 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a34004e00 con 0x7f7a44071040 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4415ccd0 con 0x7f7a44071040 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34007310 con 0x7f7a44071040 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.437+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4415d900 con 0x7f7a44071040 2023-11-28T11:55:10.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.438+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4406a270 con 0x7f7a44071040 2023-11-28T11:55:10.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.438+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a3401a020 con 0x7f7a44071040 2023-11-28T11:55:10.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.439+0000 7f7a3affd700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7a280682f0 0x7f7a2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:10.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.439+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7a340857a0 con 0x7f7a44071040 2023-11-28T11:55:10.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.439+0000 7f7a4a2eb700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7a280682f0 0x7f7a2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:10.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.439+0000 7f7a4a2eb700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7a280682f0 0x7f7a2806a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a400090b0 tx=0x7f7a4001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:10.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.443+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a34051f10 con 0x7f7a44071040 2023-11-28T11:55:10.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.665+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7a440c3d10 con 0x7f7a44071040 2023-11-28T11:55:10.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.665+0000 7f7a3affd700 1 -- 172.21.15.144:0/1545035129 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7a440c3d10 con 0x7f7a44071040 2023-11-28T11:55:10.666 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:10.666 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:10.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7a280682f0 msgr2=0x7f7a2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:10.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7a280682f0 0x7f7a2806a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a400090b0 tx=0x7f7a4001c040 comp rx=0 tx=0).stop 2023-11-28T11:55:10.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 msgr2=0x7f7a440b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:10.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a440b0e40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400c580 tx=0x7f7a3400c890 comp rx=0 tx=0).stop 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 shutdown_connections 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7a280682f0 0x7f7a2806a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a44160540 0x7f7a440cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a4406dee0 0x7f7a441f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 --2- 172.21.15.144:0/1545035129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a44071040 0x7f7a440b0e40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 >> 172.21.15.144:0/1545035129 conn(0x7f7a440d0000 msgr2=0x7f7a4415e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 shutdown_connections 2023-11-28T11:55:10.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:10.667+0000 7f7a4c54f700 1 -- 172.21.15.144:0/1545035129 wait complete. 2023-11-28T11:55:10.680 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:15.681 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:15.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.811+0000 7ff08a80b700 1 Processor -- start 2023-11-28T11:55:15.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.811+0000 7ff08a80b700 1 -- start start 2023-11-28T11:55:15.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff08a80b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff084071040 0x7ff084071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:15.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff08a80b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff08406dee0 0x7ff08406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:15.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff08a80b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff084071950 con 0x7ff08406cdc0 2023-11-28T11:55:15.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff08a80b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff084161550 con 0x7ff08406dee0 2023-11-28T11:55:15.814 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff08a80b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08415d060 con 0x7ff084071040 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff08406dee0 0x7ff08406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff08406dee0 0x7ff08406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38944/0 (socket says 172.21.15.144:38944) 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff088da8700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff08406cdc0 0x7ff08406d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55660/0 (socket says 172.21.15.144:55660) 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 -- 172.21.15.144:0/1957608153 learned_addr learned my addr 172.21.15.144:0/1957608153 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff083fff700 1 --2- 172.21.15.144:0/1957608153 >> v2:172.21.15.177:3300/0 conn(0x7ff084071040 0x7ff084071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 -- 172.21.15.144:0/1957608153 >> v2:172.21.15.177:3300/0 conn(0x7ff084071040 msgr2=0x7ff084071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 --2- 172.21.15.144:0/1957608153 >> v2:172.21.15.177:3300/0 conn(0x7ff084071040 0x7ff084071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 -- 172.21.15.144:0/1957608153 >> v1:172.21.15.144:6789/0 conn(0x7ff08406cdc0 legacy=0x7ff08406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff0837fe700 1 -- 172.21.15.144:0/1957608153 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08415d340 con 0x7ff08406dee0 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.812+0000 7ff082ffd700 1 -- 172.21.15.144:0/1957608153 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181926176 0 0) 0x7ff084071950 con 0x7ff08406cdc0 2023-11-28T11:55:15.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff0837fe700 1 --2- 172.21.15.144:0/1957608153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff08406dee0 0x7ff08406e330 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff0840d0f50 tx=0x7ff074009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608015556ff254cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff082ffd700 1 -- 172.21.15.144:0/1957608153 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400b870 con 0x7ff08406dee0 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff082ffd700 1 -- 172.21.15.144:0/1957608153 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07400b9d0 con 0x7ff08406dee0 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff082ffd700 1 -- 172.21.15.144:0/1957608153 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400bd60 con 0x7ff08406dee0 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 -- 172.21.15.144:0/1957608153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff08406dee0 msgr2=0x7ff08406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 --2- 172.21.15.144:0/1957608153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff08406dee0 0x7ff08406e330 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff0840d0f50 tx=0x7ff074009bd0 comp rx=0 tx=0).stop 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 -- 172.21.15.144:0/1957608153 shutdown_connections 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 --2- 172.21.15.144:0/1957608153 >> v2:172.21.15.177:3300/0 conn(0x7ff084071040 0x7ff084071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 --2- 172.21.15.144:0/1957608153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff08406dee0 0x7ff08406e330 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 -- 172.21.15.144:0/1957608153 >> 172.21.15.144:0/1957608153 conn(0x7ff0840d0000 msgr2=0x7ff0840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 -- 172.21.15.144:0/1957608153 shutdown_connections 2023-11-28T11:55:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.813+0000 7ff08a80b700 1 -- 172.21.15.144:0/1957608153 wait complete. 2023-11-28T11:55:15.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.815+0000 7ff08a80b700 1 Processor -- start 2023-11-28T11:55:15.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.815+0000 7ff08a80b700 1 -- start start 2023-11-28T11:55:15.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.815+0000 7ff08a80b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08406cdc0 0x7ff0841524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:15.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.815+0000 7ff08a80b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08406dee0 0x7ff0841f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:15.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff08a80b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 0x7ff084153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff08a80b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0841f6580 con 0x7ff084071040 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff08a80b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0841f6860 con 0x7ff08406cdc0 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff08a80b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840b2fb0 con 0x7ff08406dee0 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 0x7ff084153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 0x7ff084153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38954/0 (socket says 172.21.15.144:38954) 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 -- 172.21.15.144:0/3581956047 learned_addr learned my addr 172.21.15.144:0/3581956047 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff0837fe700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08406dee0 0x7ff0841f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff083fff700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08406cdc0 0x7ff0841524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 -- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08406dee0 msgr2=0x7ff0841f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08406dee0 0x7ff0841f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 -- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08406cdc0 msgr2=0x7ff0841524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:15.820 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08406cdc0 0x7ff0841524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 -- 172.21.15.144:0/3581956047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074009550 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff088da8700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 0x7ff084153b40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00cba0 tx=0x7ff07c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00ab80 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c00ad00 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c007310 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08415d900 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.816+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08415d620 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.817+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08406e870 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.818+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff07c010070 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.818+0000 7ff080ff9700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff0700682f0 0x7ff07006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.818+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff07c0856f0 con 0x7ff084071040 2023-11-28T11:55:15.821 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.818+0000 7ff083fff700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff0700682f0 0x7ff07006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:15.822 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.818+0000 7ff083fff700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff0700682f0 0x7ff07006a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff078009770 tx=0x7ff078008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:15.823 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:15.822+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff07c051e60 con 0x7ff084071040 2023-11-28T11:55:16.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.047+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff0840b34d0 con 0x7ff084071040 2023-11-28T11:55:16.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.048+0000 7ff080ff9700 1 -- 172.21.15.144:0/3581956047 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff0840b34d0 con 0x7ff084071040 2023-11-28T11:55:16.049 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:16.049 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff0700682f0 msgr2=0x7ff07006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff0700682f0 0x7ff07006a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff078009770 tx=0x7ff078008040 comp rx=0 tx=0).stop 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 msgr2=0x7ff084153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 0x7ff084153b40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00cba0 tx=0x7ff07c00cf60 comp rx=0 tx=0).stop 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 shutdown_connections 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff084071040 0x7ff084153b40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff0700682f0 0x7ff07006a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff08406cdc0 0x7ff0841524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 --2- 172.21.15.144:0/3581956047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff08406dee0 0x7ff0841f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 >> 172.21.15.144:0/3581956047 conn(0x7ff0840d0000 msgr2=0x7ff08406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:16.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 shutdown_connections 2023-11-28T11:55:16.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:16.050+0000 7ff08a80b700 1 -- 172.21.15.144:0/3581956047 wait complete. 2023-11-28T11:55:16.062 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:21.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:21.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.206+0000 7f9ba426b700 1 Processor -- start 2023-11-28T11:55:21.207 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.206+0000 7f9ba426b700 1 -- start start 2023-11-28T11:55:21.209 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba426b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c15c9b0 0x7f9b9c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba426b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b9c161340 0x7f9b9c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba426b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c0d1950 con 0x7f9b9c0d2320 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba426b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c0b5800 con 0x7f9b9c161340 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba426b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0b5520 con 0x7f9b9c15c9b0 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba1806700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b9c161340 0x7f9b9c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba1806700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b9c161340 0x7f9b9c0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38958/0 (socket says 172.21.15.144:38958) 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.207+0000 7f9ba1806700 1 -- 172.21.15.144:0/2953506680 learned_addr learned my addr 172.21.15.144:0/2953506680 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba2007700 1 --2- 172.21.15.144:0/2953506680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c15c9b0 0x7f9b9c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1806700 1 -- 172.21.15.144:0/2953506680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c15c9b0 msgr2=0x7f9b9c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1806700 1 --2- 172.21.15.144:0/2953506680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c15c9b0 0x7f9b9c0b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1806700 1 -- 172.21.15.144:0/2953506680 >> v1:172.21.15.177:6789/0 conn(0x7f9b9c0d2320 legacy=0x7f9b9c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1806700 1 -- 172.21.15.144:0/2953506680 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0d1300 con 0x7f9b9c161340 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1806700 1 --2- 172.21.15.144:0/2953506680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c0d1de0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098f0 tx=0x7f9b8c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7607322fccf8e1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1005700 1 -- 172.21.15.144:0/2953506680 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c004230 con 0x7f9b9c161340 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1005700 1 -- 172.21.15.144:0/2953506680 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c004390 con 0x7f9b9c161340 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba1005700 1 -- 172.21.15.144:0/2953506680 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0036a0 con 0x7f9b9c161340 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba426b700 1 -- 172.21.15.144:0/2953506680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 msgr2=0x7f9b9c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.208+0000 7f9ba426b700 1 --2- 172.21.15.144:0/2953506680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c0d1de0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098f0 tx=0x7f9b8c009c00 comp rx=0 tx=0).stop 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.209+0000 7f9ba426b700 1 -- 172.21.15.144:0/2953506680 shutdown_connections 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.209+0000 7f9ba426b700 1 --2- 172.21.15.144:0/2953506680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c15c9b0 0x7f9b9c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.209+0000 7f9ba426b700 1 --2- 172.21.15.144:0/2953506680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c0d1de0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.209+0000 7f9ba426b700 1 -- 172.21.15.144:0/2953506680 >> 172.21.15.144:0/2953506680 conn(0x7f9b9c072600 msgr2=0x7f9b9c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.209+0000 7f9ba426b700 1 -- 172.21.15.144:0/2953506680 shutdown_connections 2023-11-28T11:55:21.211 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.209+0000 7f9ba426b700 1 -- 172.21.15.144:0/2953506680 wait complete. 2023-11-28T11:55:21.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.210+0000 7f9ba426b700 1 Processor -- start 2023-11-28T11:55:21.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.210+0000 7f9ba426b700 1 -- start start 2023-11-28T11:55:21.214 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c0d2320 0x7f9b9c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c15c9b0 0x7f9b9c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c154d20 con 0x7f9b9c161340 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c157050 con 0x7f9b9c15c9b0 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c154f00 con 0x7f9b9c0d2320 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38968/0 (socket says 172.21.15.144:38968) 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 -- 172.21.15.144:0/552645564 learned_addr learned my addr 172.21.15.144:0/552645564 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba1806700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c15c9b0 0x7f9b9c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2007700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c0d2320 0x7f9b9c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 -- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c0d2320 msgr2=0x7f9b9c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:21.215 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c0d2320 0x7f9b9c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 -- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c15c9b0 msgr2=0x7f9b9c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c15c9b0 0x7f9b9c156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 -- 172.21.15.144:0/552645564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009550 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba1806700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c15c9b0 0x7f9b9c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba2808700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c15ecd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400b970 tx=0x7f9b9400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400c9b0 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400cb10 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94014440 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c0b5130 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.211+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c0d15e0 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.212+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c0b52c0 con 0x7f9b9c161340 2023-11-28T11:55:21.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.213+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b9401b030 con 0x7f9b9c161340 2023-11-28T11:55:21.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.213+0000 7f9b92ffd700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b800682f0 0x7f9b8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:21.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.213+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9b94085860 con 0x7f9b9c161340 2023-11-28T11:55:21.217 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.213+0000 7f9ba2007700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b800682f0 0x7f9b8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:21.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.213+0000 7f9ba2007700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b800682f0 0x7f9b8006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800a990 tx=0x7f9b98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:21.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.217+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9b94052030 con 0x7f9b9c161340 2023-11-28T11:55:21.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.444+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9b9c0745c0 con 0x7f9b9c161340 2023-11-28T11:55:21.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.444+0000 7f9b92ffd700 1 -- 172.21.15.144:0/552645564 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9b94055650 con 0x7f9b9c161340 2023-11-28T11:55:21.445 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:21.446 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:21.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b800682f0 msgr2=0x7f9b8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:21.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b800682f0 0x7f9b8006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800a990 tx=0x7f9b98008040 comp rx=0 tx=0).stop 2023-11-28T11:55:21.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 msgr2=0x7f9b9c15ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c15ecd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400b970 tx=0x7f9b9400bd30 comp rx=0 tx=0).stop 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 shutdown_connections 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b9c161340 0x7f9b9c15ecd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9b800682f0 0x7f9b8006a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9b9c0d2320 0x7f9b9c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 --2- 172.21.15.144:0/552645564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b9c15c9b0 0x7f9b9c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 >> 172.21.15.144:0/552645564 conn(0x7f9b9c072600 msgr2=0x7f9b9c15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 shutdown_connections 2023-11-28T11:55:21.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:21.446+0000 7f9ba426b700 1 -- 172.21.15.144:0/552645564 wait complete. 2023-11-28T11:55:21.459 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:23.212 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:55:23.215 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:55:26.460 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:26.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.592+0000 7fe488891700 1 Processor -- start 2023-11-28T11:55:26.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.593+0000 7fe488891700 1 -- start start 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.593+0000 7fe488891700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe48406dee0 0x7fe48406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.593+0000 7fe488891700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe484160540 0x7fe484160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.593+0000 7fe488891700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48415d060 con 0x7fe484071040 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.593+0000 7fe488891700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48415d340 con 0x7fe484160540 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.593+0000 7fe488891700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48415d620 con 0x7fe48406dee0 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe484160540 0x7fe484160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:26.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe484160540 0x7fe484160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38972/0 (socket says 172.21.15.144:38972) 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 -- 172.21.15.144:0/904505543 learned_addr learned my addr 172.21.15.144:0/904505543 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe481d9b700 1 --2- 172.21.15.144:0/904505543 >> v2:172.21.15.177:3300/0 conn(0x7fe48406dee0 0x7fe48406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 -- 172.21.15.144:0/904505543 >> v2:172.21.15.177:3300/0 conn(0x7fe48406dee0 msgr2=0x7fe48406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 --2- 172.21.15.144:0/904505543 >> v2:172.21.15.177:3300/0 conn(0x7fe48406dee0 0x7fe48406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 -- 172.21.15.144:0/904505543 >> v1:172.21.15.177:6789/0 conn(0x7fe484071040 legacy=0x7fe484071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 -- 172.21.15.144:0/904505543 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48415d900 con 0x7fe484160540 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe482d9d700 1 --2- 172.21.15.144:0/904505543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe484160980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe48406fb00 tx=0x7fe47c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4f12245124ab29 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:26.596 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe48159a700 1 -- 172.21.15.144:0/904505543 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00e070 con 0x7fe484160540 2023-11-28T11:55:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe48159a700 1 -- 172.21.15.144:0/904505543 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c003a20 con 0x7fe484160540 2023-11-28T11:55:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.594+0000 7fe48159a700 1 -- 172.21.15.144:0/904505543 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c003dd0 con 0x7fe484160540 2023-11-28T11:55:26.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 -- 172.21.15.144:0/904505543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 msgr2=0x7fe484160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:26.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 --2- 172.21.15.144:0/904505543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe484160980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe48406fb00 tx=0x7fe47c00b330 comp rx=0 tx=0).stop 2023-11-28T11:55:26.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 -- 172.21.15.144:0/904505543 shutdown_connections 2023-11-28T11:55:26.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 --2- 172.21.15.144:0/904505543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe484160980 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 --2- 172.21.15.144:0/904505543 >> v2:172.21.15.177:3300/0 conn(0x7fe48406dee0 0x7fe48406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 -- 172.21.15.144:0/904505543 >> 172.21.15.144:0/904505543 conn(0x7fe4840d0000 msgr2=0x7fe4840d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:26.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 -- 172.21.15.144:0/904505543 shutdown_connections 2023-11-28T11:55:26.605 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.595+0000 7fe488891700 1 -- 172.21.15.144:0/904505543 wait complete. 2023-11-28T11:55:26.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.597+0000 7fe488891700 1 Processor -- start 2023-11-28T11:55:26.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.597+0000 7fe488891700 1 -- start start 2023-11-28T11:55:26.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe48406dee0 0x7fe4840c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:26.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe484071040 0x7fe4840c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:26.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe4840c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:26.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48414e9e0 con 0x7fe484160540 2023-11-28T11:55:26.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840b3210 con 0x7fe48406dee0 2023-11-28T11:55:26.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840b34f0 con 0x7fe484071040 2023-11-28T11:55:26.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe4840c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:26.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe4840c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38980/0 (socket says 172.21.15.144:38980) 2023-11-28T11:55:26.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 -- 172.21.15.144:0/4100325208 learned_addr learned my addr 172.21.15.144:0/4100325208 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:26.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe48259c700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe48406dee0 0x7fe4840c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:26.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe481d9b700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe484071040 0x7fe4840c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:26.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 -- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe484071040 msgr2=0x7fe4840c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:26.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe484071040 0x7fe4840c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 -- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe48406dee0 msgr2=0x7fe4840c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:26.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe48406dee0 0x7fe4840c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 -- 172.21.15.144:0/4100325208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c009b30 con 0x7fe484160540 2023-11-28T11:55:26.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe48259c700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe48406dee0 0x7fe4840c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:55:26.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe482d9d700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe4840c3fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe48406c7b0 tx=0x7fe47c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:26.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c004440 con 0x7fe484160540 2023-11-28T11:55:26.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0045a0 con 0x7fe484160540 2023-11-28T11:55:26.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48415ccd0 con 0x7fe484160540 2023-11-28T11:55:26.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c004930 con 0x7fe484160540 2023-11-28T11:55:26.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.598+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48406e870 con 0x7fe484160540 2023-11-28T11:55:26.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.599+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48406a270 con 0x7fe484160540 2023-11-28T11:55:26.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.600+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe47c022030 con 0x7fe484160540 2023-11-28T11:55:26.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.600+0000 7fe472ffd700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4640682f0 0x7fe46406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:26.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.600+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe47c087170 con 0x7fe484160540 2023-11-28T11:55:26.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.600+0000 7fe48259c700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4640682f0 0x7fe46406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:26.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.601+0000 7fe48259c700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4640682f0 0x7fe46406a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe474006fd0 tx=0x7fe474008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:26.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.604+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe47c0538c0 con 0x7fe484160540 2023-11-28T11:55:26.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.829+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe4840720a0 con 0x7fe484160540 2023-11-28T11:55:26.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.830+0000 7fe472ffd700 1 -- 172.21.15.144:0/4100325208 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe47c056ee0 con 0x7fe484160540 2023-11-28T11:55:26.830 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:26.830 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4640682f0 msgr2=0x7fe46406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4640682f0 0x7fe46406a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe474006fd0 tx=0x7fe474008040 comp rx=0 tx=0).stop 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 msgr2=0x7fe4840c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe4840c3fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe48406c7b0 tx=0x7fe47c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 shutdown_connections 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe484160540 0x7fe4840c3fe0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe4640682f0 0x7fe46406a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe48406dee0 0x7fe4840c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 --2- 172.21.15.144:0/4100325208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe484071040 0x7fe4840c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 >> 172.21.15.144:0/4100325208 conn(0x7fe4840d0000 msgr2=0x7fe48406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:26.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 shutdown_connections 2023-11-28T11:55:26.834 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:26.832+0000 7fe488891700 1 -- 172.21.15.144:0/4100325208 wait complete. 2023-11-28T11:55:26.846 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:31.848 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:31.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.979+0000 7f57a38fb700 1 Processor -- start 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.979+0000 7f57a38fb700 1 -- start start 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a38fb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f579c071040 0x7f579c071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a38fb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f579c06dee0 0x7f579c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a38fb700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c071950 con 0x7f579c06cdc0 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a38fb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c161550 con 0x7f579c06dee0 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a38fb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c15d060 con 0x7f579c071040 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a0e96700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f579c06dee0 0x7f579c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a1e98700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f579c06cdc0 0x7f579c06d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55700/0 (socket says 172.21.15.144:55700) 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.980+0000 7f57a1e98700 1 -- 172.21.15.144:0/2519326344 learned_addr learned my addr 172.21.15.144:0/2519326344 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a1697700 1 --2- 172.21.15.144:0/2519326344 >> v2:172.21.15.177:3300/0 conn(0x7f579c071040 0x7f579c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:31.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a0e96700 1 -- 172.21.15.144:0/2519326344 >> v2:172.21.15.177:3300/0 conn(0x7f579c071040 msgr2=0x7f579c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a0e96700 1 --2- 172.21.15.144:0/2519326344 >> v2:172.21.15.177:3300/0 conn(0x7f579c071040 0x7f579c071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a0e96700 1 -- 172.21.15.144:0/2519326344 >> v1:172.21.15.144:6789/0 conn(0x7f579c06cdc0 legacy=0x7f579c06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f5793fff700 1 -- 172.21.15.144:0/2519326344 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506585534 0 0) 0x7f579c071950 con 0x7f579c06cdc0 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a0e96700 1 -- 172.21.15.144:0/2519326344 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c15d340 con 0x7f579c06dee0 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a0e96700 1 --2- 172.21.15.144:0/2519326344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06dee0 0x7f579c06e330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f579c0d0f50 tx=0x7f578c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4623ce0498ff526 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f5793fff700 1 -- 172.21.15.144:0/2519326344 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c011830 con 0x7f579c06dee0 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f5793fff700 1 -- 172.21.15.144:0/2519326344 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c011990 con 0x7f579c06dee0 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f5793fff700 1 -- 172.21.15.144:0/2519326344 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c011d20 con 0x7f579c06dee0 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.981+0000 7f57a38fb700 1 -- 172.21.15.144:0/2519326344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06dee0 msgr2=0x7f579c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 --2- 172.21.15.144:0/2519326344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06dee0 0x7f579c06e330 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f579c0d0f50 tx=0x7f578c009b20 comp rx=0 tx=0).stop 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 -- 172.21.15.144:0/2519326344 shutdown_connections 2023-11-28T11:55:31.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 --2- 172.21.15.144:0/2519326344 >> v2:172.21.15.177:3300/0 conn(0x7f579c071040 0x7f579c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:31.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 --2- 172.21.15.144:0/2519326344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06dee0 0x7f579c06e330 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:31.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 -- 172.21.15.144:0/2519326344 >> 172.21.15.144:0/2519326344 conn(0x7f579c0d0000 msgr2=0x7f579c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:31.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 -- 172.21.15.144:0/2519326344 shutdown_connections 2023-11-28T11:55:31.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.982+0000 7f57a38fb700 1 -- 172.21.15.144:0/2519326344 wait complete. 2023-11-28T11:55:31.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.983+0000 7f57a38fb700 1 Processor -- start 2023-11-28T11:55:31.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.983+0000 7f57a38fb700 1 -- start start 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 0x7f579c0c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f579c06dee0 0x7f579c15e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f579c071040 0x7f579c1e6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b5a80 con 0x7f579c06cdc0 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0c3fe0 con 0x7f579c06dee0 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0c4210 con 0x7f579c071040 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 0x7f579c0c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 0x7f579c0c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38990/0 (socket says 172.21.15.144:38990) 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 -- 172.21.15.144:0/947194368 learned_addr learned my addr 172.21.15.144:0/947194368 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a0e96700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f579c06dee0 0x7f579c15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:31.989 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1e98700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f579c071040 0x7f579c1e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 -- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f579c071040 msgr2=0x7f579c1e6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f579c071040 0x7f579c1e6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 -- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f579c06dee0 msgr2=0x7f579c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f579c06dee0 0x7f579c15e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 -- 172.21.15.144:0/947194368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0094a0 con 0x7f579c06cdc0 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1e98700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f579c071040 0x7f579c1e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a1697700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 0x7f579c0c3aa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f579800d800 tx=0x7f579800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980125f0 con 0x7f579c06cdc0 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798012750 con 0x7f579c06cdc0 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c15d900 con 0x7f579c06cdc0 2023-11-28T11:55:31.990 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.984+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798012b00 con 0x7f579c06cdc0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.985+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c15d620 con 0x7f579c06cdc0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.985+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c06e870 con 0x7f579c06cdc0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.986+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5798012c60 con 0x7f579c06cdc0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.986+0000 7f5791ffb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f57800682f0 0x7f578006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.986+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f5798086330 con 0x7f579c06cdc0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.986+0000 7f57a0e96700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f57800682f0 0x7f578006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.987+0000 7f57a0e96700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f57800682f0 0x7f578006a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f578c0090b0 tx=0x7f578c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:31.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:31.990+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5798052a10 con 0x7f579c06cdc0 2023-11-28T11:55:32.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.215+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f579c0c5100 con 0x7f579c06cdc0 2023-11-28T11:55:32.216 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.215+0000 7f5791ffb700 1 -- 172.21.15.144:0/947194368 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5798056030 con 0x7f579c06cdc0 2023-11-28T11:55:32.216 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:32.216 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:32.218 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.217+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f57800682f0 msgr2=0x7f578006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.217+0000 7f57a38fb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f57800682f0 0x7f578006a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f578c0090b0 tx=0x7f578c01c040 comp rx=0 tx=0).stop 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.217+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 msgr2=0x7f579c0c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.217+0000 7f57a38fb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 0x7f579c0c3aa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f579800d800 tx=0x7f579800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 shutdown_connections 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f579c06cdc0 0x7f579c0c3aa0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f57800682f0 0x7f578006a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f579c071040 0x7f579c1e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 --2- 172.21.15.144:0/947194368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f579c06dee0 0x7f579c15e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 >> 172.21.15.144:0/947194368 conn(0x7f579c0d0000 msgr2=0x7f579c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 shutdown_connections 2023-11-28T11:55:32.219 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:32.218+0000 7f57a38fb700 1 -- 172.21.15.144:0/947194368 wait complete. 2023-11-28T11:55:32.230 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:37.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:37.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.363+0000 7f836eeea700 1 Processor -- start 2023-11-28T11:55:37.364 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.363+0000 7f836eeea700 1 -- start start 2023-11-28T11:55:37.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836eeea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f836815a9f0 0x7f83680cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836eeea700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8368071010 0x7f836806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836eeea700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8368071d80 0x7f836806a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.366 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836eeea700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836815f470 con 0x7f8368071010 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836eeea700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83680cd880 con 0x7f8368071d80 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836eeea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836815eef0 con 0x7f836815a9f0 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8368071010 0x7f836806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8368071010 0x7f836806a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38998/0 (socket says 172.21.15.144:38998) 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 -- 172.21.15.144:0/608905079 learned_addr learned my addr 172.21.15.144:0/608905079 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836cc86700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f836815a9f0 0x7f83680cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836d487700 1 --2- 172.21.15.144:0/608905079 >> v2:172.21.15.177:3300/0 conn(0x7f8368071d80 0x7f836806a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 -- 172.21.15.144:0/608905079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f836815a9f0 msgr2=0x7f83680cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f836815a9f0 0x7f83680cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 -- 172.21.15.144:0/608905079 >> v2:172.21.15.177:3300/0 conn(0x7f8368071d80 msgr2=0x7f836806a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:37.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 --2- 172.21.15.144:0/608905079 >> v2:172.21.15.177:3300/0 conn(0x7f8368071d80 0x7f836806a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 -- 172.21.15.144:0/608905079 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836815adc0 con 0x7f8368071010 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836d487700 1 --2- 172.21.15.144:0/608905079 >> v2:172.21.15.177:3300/0 conn(0x7f8368071d80 0x7f836806a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f836cc86700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f836815a9f0 0x7f83680cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.364+0000 7f8367fff700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8368071010 0x7f836806a270 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f83680cfc90 tx=0x7f8354009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9bab9959d08a69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f83677fe700 1 -- 172.21.15.144:0/608905079 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835400b870 con 0x7f8368071010 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f83677fe700 1 -- 172.21.15.144:0/608905079 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835400b9d0 con 0x7f8368071010 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f83677fe700 1 -- 172.21.15.144:0/608905079 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835400bd60 con 0x7f8368071010 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 -- 172.21.15.144:0/608905079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8368071010 msgr2=0x7f836806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8368071010 0x7f836806a270 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f83680cfc90 tx=0x7f8354009bd0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 -- 172.21.15.144:0/608905079 shutdown_connections 2023-11-28T11:55:37.368 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 --2- 172.21.15.144:0/608905079 >> v2:172.21.15.177:3300/0 conn(0x7f8368071d80 0x7f836806a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f836815a9f0 0x7f83680cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 --2- 172.21.15.144:0/608905079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8368071010 0x7f836806a270 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 -- 172.21.15.144:0/608905079 >> 172.21.15.144:0/608905079 conn(0x7f836806b380 msgr2=0x7f836806b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:37.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 -- 172.21.15.144:0/608905079 shutdown_connections 2023-11-28T11:55:37.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.365+0000 7f836eeea700 1 -- 172.21.15.144:0/608905079 wait complete. 2023-11-28T11:55:37.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.367+0000 7f836eeea700 1 Processor -- start 2023-11-28T11:55:37.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.367+0000 7f836eeea700 1 -- start start 2023-11-28T11:55:37.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.367+0000 7f836eeea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8368071010 0x7f836815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.367+0000 7f836eeea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8368071d80 0x7f836815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.367+0000 7f836eeea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 0x7f8368145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836eeea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836815b520 con 0x7f836815a9f0 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836eeea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83681593f0 con 0x7f8368071d80 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836eeea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83680c7f60 con 0x7f8368071010 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 0x7f8368145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 0x7f8368145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39006/0 (socket says 172.21.15.144:39006) 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836cc86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8368071010 0x7f836815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 -- 172.21.15.144:0/1820924281 learned_addr learned my addr 172.21.15.144:0/1820924281 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f8367fff700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8368071d80 0x7f836815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 -- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8368071010 msgr2=0x7f836815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:37.372 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8368071010 0x7f836815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 -- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8368071d80 msgr2=0x7f836815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8368071d80 0x7f836815cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 -- 172.21.15.144:0/1820924281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8354009550 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836d487700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 0x7f8368145bf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8360007e10 tx=0x7f836000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836000dd20 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836000f040 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836806d900 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360003c60 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.368+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836815f190 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.369+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836806dbe0 con 0x7f836815a9f0 2023-11-28T11:55:37.373 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.370+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f836001c030 con 0x7f836815a9f0 2023-11-28T11:55:37.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.370+0000 7f83657fa700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f834c0682f0 0x7f834c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:37.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.370+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8360085c40 con 0x7f836815a9f0 2023-11-28T11:55:37.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.370+0000 7f836cc86700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f834c0682f0 0x7f834c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:37.374 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.371+0000 7f836cc86700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f834c0682f0 0x7f834c06a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c940 tx=0x7f835c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:37.375 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.375+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8360052380 con 0x7f836815a9f0 2023-11-28T11:55:37.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.598+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83680745c0 con 0x7f836815a9f0 2023-11-28T11:55:37.599 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.598+0000 7f83657fa700 1 -- 172.21.15.144:0/1820924281 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f836001a080 con 0x7f836815a9f0 2023-11-28T11:55:37.600 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:37.600 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:37.601 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f834c0682f0 msgr2=0x7f834c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:37.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f834c0682f0 0x7f834c06a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c940 tx=0x7f835c008040 comp rx=0 tx=0).stop 2023-11-28T11:55:37.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 msgr2=0x7f8368145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:37.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 0x7f8368145bf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8360007e10 tx=0x7f836000d3b0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 shutdown_connections 2023-11-28T11:55:37.602 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836815a9f0 0x7f8368145bf0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f834c0682f0 0x7f834c06a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8368071010 0x7f836815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 --2- 172.21.15.144:0/1820924281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8368071d80 0x7f836815cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:37.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 >> 172.21.15.144:0/1820924281 conn(0x7f836806b380 msgr2=0x7f83680b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:37.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 shutdown_connections 2023-11-28T11:55:37.604 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:37.600+0000 7f836eeea700 1 -- 172.21.15.144:0/1820924281 wait complete. 2023-11-28T11:55:37.613 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:42.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:42.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.745+0000 7fc22cccf700 1 Processor -- start 2023-11-28T11:55:42.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.746+0000 7fc22cccf700 1 -- start start 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.746+0000 7fc22cccf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.746+0000 7fc22cccf700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc228160540 0x7fc2281609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.746+0000 7fc22cccf700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22815d060 con 0x7fc228071040 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.746+0000 7fc22cccf700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22815d340 con 0x7fc228160540 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.746+0000 7fc22cccf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22815d620 con 0x7fc22806dee0 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22659c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fc228071040 0x7fc228071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55722/0 (socket says 172.21.15.144:55722) 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22659c700 1 -- 172.21.15.144:0/1997435633 learned_addr learned my addr 172.21.15.144:0/1997435633 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc226d9d700 1 --2- 172.21.15.144:0/1997435633 >> v2:172.21.15.144:3300/0 conn(0x7fc228160540 0x7fc2281609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc225d9b700 1 --2- 172.21.15.144:0/1997435633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:42.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22559a700 1 -- 172.21.15.144:0/1997435633 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173618296 0 0) 0x7fc22815d060 con 0x7fc228071040 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22559a700 1 -- 172.21.15.144:0/1997435633 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2140035b0 con 0x7fc228071040 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc226d9d700 1 -- 172.21.15.144:0/1997435633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 msgr2=0x7fc22806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc226d9d700 1 --2- 172.21.15.144:0/1997435633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc226d9d700 1 -- 172.21.15.144:0/1997435633 >> v1:172.21.15.144:6789/0 conn(0x7fc228071040 legacy=0x7fc228071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc226d9d700 1 -- 172.21.15.144:0/1997435633 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22815d900 con 0x7fc228160540 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc226d9d700 1 --2- 172.21.15.144:0/1997435633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228160540 0x7fc2281609c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc220009d90 tx=0x7fc22000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959ce94c0abfb8fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22559a700 1 -- 172.21.15.144:0/1997435633 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000e070 con 0x7fc228160540 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22559a700 1 -- 172.21.15.144:0/1997435633 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22000bb20 con 0x7fc228160540 2023-11-28T11:55:42.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.747+0000 7fc22559a700 1 -- 172.21.15.144:0/1997435633 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220012430 con 0x7fc228160540 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 -- 172.21.15.144:0/1997435633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228160540 msgr2=0x7fc2281609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 --2- 172.21.15.144:0/1997435633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228160540 0x7fc2281609c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc220009d90 tx=0x7fc22000b330 comp rx=0 tx=0).stop 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 -- 172.21.15.144:0/1997435633 shutdown_connections 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 --2- 172.21.15.144:0/1997435633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228160540 0x7fc2281609c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 --2- 172.21.15.144:0/1997435633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 -- 172.21.15.144:0/1997435633 >> 172.21.15.144:0/1997435633 conn(0x7fc2280d0000 msgr2=0x7fc2280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 -- 172.21.15.144:0/1997435633 shutdown_connections 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.748+0000 7fc22cccf700 1 -- 172.21.15.144:0/1997435633 wait complete. 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.749+0000 7fc22cccf700 1 Processor -- start 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 -- start start 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 0x7fc2280c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:42.755 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc228160540 0x7fc2280c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281f69a0 con 0x7fc228071040 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280b2ed0 con 0x7fc228160540 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22cccf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280b31b0 con 0x7fc22806dee0 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc225d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 0x7fc2280c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc225d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 0x7fc2280c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39018/0 (socket says 172.21.15.144:39018) 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc225d9b700 1 -- 172.21.15.144:0/3131741194 learned_addr learned my addr 172.21.15.144:0/3131741194 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc22659c700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.750+0000 7fc226d9d700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc228160540 0x7fc2280c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc225d9b700 1 -- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 msgr2=0x7fc22814db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc225d9b700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22814db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc225d9b700 1 -- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc228160540 msgr2=0x7fc2280c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc225d9b700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc228160540 0x7fc2280c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc225d9b700 1 -- 172.21.15.144:0/3131741194 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220009a70 con 0x7fc228071040 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc225d9b700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 0x7fc2280c5de0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc21800e870 tx=0x7fc21800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218009590 con 0x7fc228071040 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218013680 con 0x7fc228071040 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22815ccd0 con 0x7fc228071040 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180139f0 con 0x7fc228071040 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.751+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22806e870 con 0x7fc228071040 2023-11-28T11:55:42.757 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.752+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22806a270 con 0x7fc228071040 2023-11-28T11:55:42.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.753+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc218019030 con 0x7fc228071040 2023-11-28T11:55:42.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.753+0000 7fc21effd700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2080682f0 0x7fc20806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:42.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.754+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc2180861b0 con 0x7fc228071040 2023-11-28T11:55:42.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.754+0000 7fc22659c700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2080682f0 0x7fc20806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:42.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.754+0000 7fc22659c700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2080682f0 0x7fc20806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc210007fd0 tx=0x7fc210007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:42.758 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.757+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc218056450 con 0x7fc228071040 2023-11-28T11:55:42.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.982+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc2280720a0 con 0x7fc228071040 2023-11-28T11:55:42.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.982+0000 7fc21effd700 1 -- 172.21.15.144:0/3131741194 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc218056040 con 0x7fc228071040 2023-11-28T11:55:42.983 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:42.983 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2080682f0 msgr2=0x7fc20806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2080682f0 0x7fc20806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc210007fd0 tx=0x7fc210007dc0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 msgr2=0x7fc2280c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 0x7fc2280c5de0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc21800e870 tx=0x7fc21800eb80 comp rx=0 tx=0).stop 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 shutdown_connections 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fc2080682f0 0x7fc20806a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc228160540 0x7fc2280c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc22806dee0 0x7fc22814db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 --2- 172.21.15.144:0/3131741194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc228071040 0x7fc2280c5de0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 >> 172.21.15.144:0/3131741194 conn(0x7fc2280d0000 msgr2=0x7fc2280cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 shutdown_connections 2023-11-28T11:55:42.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:42.984+0000 7fc22cccf700 1 -- 172.21.15.144:0/3131741194 wait complete. 2023-11-28T11:55:42.997 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:47.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:48.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.128+0000 7f470a0f1700 1 Processor -- start 2023-11-28T11:55:48.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.128+0000 7f470a0f1700 1 -- start start 2023-11-28T11:55:48.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f470a0f1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f47040ae9d0 0x7f470415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f470a0f1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f470406a5e0 0x7f470406a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f470a0f1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040aee40 con 0x7f470406e500 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f470a0f1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470415bf10 con 0x7f47040ae9d0 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f470a0f1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470406c930 con 0x7f470406a5e0 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4702ffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f47040ae9d0 0x7f470415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f47037fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f470406e500 0x7f470406e8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55736/0 (socket says 172.21.15.144:55736) 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f47037fe700 1 -- 172.21.15.144:0/1096422022 learned_addr learned my addr 172.21.15.144:0/1096422022 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4702ffd700 1 --2- 172.21.15.144:0/1096422022 >> v2:172.21.15.144:3300/0 conn(0x7f47040ae9d0 0x7f470415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39026/0 (socket says 172.21.15.144:39026) 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4703fff700 1 --2- 172.21.15.144:0/1096422022 >> v2:172.21.15.177:3300/0 conn(0x7f470406a5e0 0x7f470406a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:48.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4702ffd700 1 -- 172.21.15.144:0/1096422022 >> v2:172.21.15.177:3300/0 conn(0x7f470406a5e0 msgr2=0x7f470406a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4702ffd700 1 --2- 172.21.15.144:0/1096422022 >> v2:172.21.15.177:3300/0 conn(0x7f470406a5e0 0x7f470406a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4702ffd700 1 -- 172.21.15.144:0/1096422022 >> v1:172.21.15.144:6789/0 conn(0x7f470406e500 legacy=0x7f470406e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f47027fc700 1 -- 172.21.15.144:0/1096422022 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391807565 0 0) 0x7f47040aee40 con 0x7f470406e500 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4702ffd700 1 -- 172.21.15.144:0/1096422022 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470406cc10 con 0x7f47040ae9d0 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.129+0000 7f4703fff700 1 --2- 172.21.15.144:0/1096422022 >> v2:172.21.15.177:3300/0 conn(0x7f470406a5e0 0x7f470406a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f4702ffd700 1 --2- 172.21.15.144:0/1096422022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47040ae9d0 0x7f470415b9d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4704072160 tx=0x7f46f4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fad6b1585f38ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f47027fc700 1 -- 172.21.15.144:0/1096422022 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4011830 con 0x7f47040ae9d0 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f47027fc700 1 -- 172.21.15.144:0/1096422022 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f4011990 con 0x7f47040ae9d0 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f47027fc700 1 -- 172.21.15.144:0/1096422022 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4011d20 con 0x7f47040ae9d0 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 -- 172.21.15.144:0/1096422022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47040ae9d0 msgr2=0x7f470415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:48.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 --2- 172.21.15.144:0/1096422022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47040ae9d0 0x7f470415b9d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4704072160 tx=0x7f46f4009b20 comp rx=0 tx=0).stop 2023-11-28T11:55:48.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 -- 172.21.15.144:0/1096422022 shutdown_connections 2023-11-28T11:55:48.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 --2- 172.21.15.144:0/1096422022 >> v2:172.21.15.177:3300/0 conn(0x7f470406a5e0 0x7f470406a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 --2- 172.21.15.144:0/1096422022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47040ae9d0 0x7f470415b9d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 -- 172.21.15.144:0/1096422022 >> 172.21.15.144:0/1096422022 conn(0x7f47040b12b0 msgr2=0x7f47040b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:48.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 -- 172.21.15.144:0/1096422022 shutdown_connections 2023-11-28T11:55:48.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.130+0000 7f470a0f1700 1 -- 172.21.15.144:0/1096422022 wait complete. 2023-11-28T11:55:48.137 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.132+0000 7f470a0f1700 1 Processor -- start 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.132+0000 7f470a0f1700 1 -- start start 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.132+0000 7f470a0f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 0x7f470415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.132+0000 7f470a0f1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470406e500 0x7f470415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f470a0f1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47040ae9d0 0x7f4704149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f470a0f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47041479c0 con 0x7f470406a5e0 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f470a0f1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47041476b0 con 0x7f47040ae9d0 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f470a0f1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47040c9660 con 0x7f470406e500 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 0x7f470415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 0x7f470415eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39030/0 (socket says 172.21.15.144:39030) 2023-11-28T11:55:48.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 -- 172.21.15.144:0/895529026 learned_addr learned my addr 172.21.15.144:0/895529026 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 -- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470406e500 msgr2=0x7f470415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470406e500 0x7f470415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 -- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47040ae9d0 msgr2=0x7f4704149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f4703fff700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47040ae9d0 0x7f4704149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47040ae9d0 0x7f4704149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 -- 172.21.15.144:0/895529026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40094a0 con 0x7f470406a5e0 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f4703fff700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47040ae9d0 0x7f4704149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f47037fe700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 0x7f470415eae0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0011e0 tx=0x7f46ec00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec004c80 con 0x7f470406a5e0 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec004e00 con 0x7f470406a5e0 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470406cef0 con 0x7f470406a5e0 2023-11-28T11:55:48.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec007310 con 0x7f470406a5e0 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.133+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47040664c0 con 0x7f470406a5e0 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.134+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470415ac80 con 0x7f470406a5e0 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.135+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46ec01a020 con 0x7f470406a5e0 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.135+0000 7f46fbfff700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f46e40682f0 0x7f46e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.135+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f46ec0859f0 con 0x7f470406a5e0 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.135+0000 7f4702ffd700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f46e40682f0 0x7f46e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:48.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.136+0000 7f4702ffd700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f46e40682f0 0x7f46e406a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4704072160 tx=0x7f46f4010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:48.141 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.140+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46ec087e60 con 0x7f470406a5e0 2023-11-28T11:55:48.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.366+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4704147b00 con 0x7f470406a5e0 2023-11-28T11:55:48.367 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.366+0000 7f46fbfff700 1 -- 172.21.15.144:0/895529026 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f46ec0521c0 con 0x7f470406a5e0 2023-11-28T11:55:48.368 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:48.368 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f46e40682f0 msgr2=0x7f46e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f46e40682f0 0x7f46e406a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4704072160 tx=0x7f46f4010000 comp rx=0 tx=0).stop 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 msgr2=0x7f470415eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 0x7f470415eae0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0011e0 tx=0x7f46ec00cae0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 shutdown_connections 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f470406a5e0 0x7f470415eae0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f46e40682f0 0x7f46e406a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.370 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47040ae9d0 0x7f4704149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 --2- 172.21.15.144:0/895529026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f470406e500 0x7f470415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:48.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 >> 172.21.15.144:0/895529026 conn(0x7f47040b12b0 msgr2=0x7f4704159f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:48.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 shutdown_connections 2023-11-28T11:55:48.371 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:48.368+0000 7f470a0f1700 1 -- 172.21.15.144:0/895529026 wait complete. 2023-11-28T11:55:48.382 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:53.295 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:55:53.298 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:55:53.382 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:53.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.508+0000 7f50ff357700 1 Processor -- start 2023-11-28T11:55:53.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.509+0000 7f50ff357700 1 -- start start 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.509+0000 7f50ff357700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 0x7f50f8140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50ff357700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f50f806f850 0x7f50f806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50ff357700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f813ff00 con 0x7f50f80cdc90 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50ff357700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f81404c0 con 0x7f50f806f850 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50ff357700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f815aa80 con 0x7f50f80cd8c0 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50fc8f2700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f50f80cdc90 0x7f50f80b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55750/0 (socket says 172.21.15.144:55750) 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50fc8f2700 1 -- 172.21.15.144:0/795959468 learned_addr learned my addr 172.21.15.144:0/795959468 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036815523 0 0) 0x7f50f813ff00 con 0x7f50f80cdc90 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50fd8f4700 1 --2- 172.21.15.144:0/795959468 >> v2:172.21.15.177:3300/0 conn(0x7f50f806f850 0x7f50f806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50fd0f3700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 0x7f50f8140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e40035b0 con 0x7f50f80cdc90 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4076212725 0 0) 0x7f50e40035b0 con 0x7f50f80cdc90 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 msgr2=0x7f50f8140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 0x7f50f8140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 >> v2:172.21.15.177:3300/0 conn(0x7f50f806f850 msgr2=0x7f50f806fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 --2- 172.21.15.144:0/795959468 >> v2:172.21.15.177:3300/0 conn(0x7f50f806f850 0x7f50f806fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80b3040 con 0x7f50f80cdc90 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f50f4001670 con 0x7f50f80cdc90 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 >> v1:172.21.15.144:6789/0 conn(0x7f50f80cdc90 legacy=0x7f50f80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 0x7f50e4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e4003f80 0x7f50e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4004d90 0x7f50e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f81404c0 con 0x7f50e4003f80 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f815aa80 con 0x7f50e4004d90 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e400ba40 con 0x7f50f80cd8c0 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e4003f80 0x7f50e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.510+0000 7f50fd0f3700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 0x7f50e4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fd8f4700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4004d90 0x7f50e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fc8f2700 1 -- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 msgr2=0x7f50e4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80cd8c0 0x7f50e4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fc8f2700 1 -- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4004d90 msgr2=0x7f50e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:53.517 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4004d90 0x7f50e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fc8f2700 1 -- 172.21.15.144:0/795959468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40035b0 con 0x7f50e4003f80 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e4003f80 0x7f50e4004850 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f50f80cea30 tx=0x7f50e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77eb1ab678a2df81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f400cda0 con 0x7f50e4003f80 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f4005960 con 0x7f50e4003f80 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50effff700 1 -- 172.21.15.144:0/795959468 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4005d10 con 0x7f50e4003f80 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50ff357700 1 -- 172.21.15.144:0/795959468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e4003f80 msgr2=0x7f50e4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50ff357700 1 --2- 172.21.15.144:0/795959468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50e4003f80 0x7f50e4004850 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f50f80cea30 tx=0x7f50e4005880 comp rx=0 tx=0).stop 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.511+0000 7f50fd8f4700 1 -- 172.21.15.144:0/795959468 reap_dead start 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.512+0000 7f50ff357700 1 -- 172.21.15.144:0/795959468 shutdown_connections 2023-11-28T11:55:53.518 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.512+0000 7f50ff357700 1 -- 172.21.15.144:0/795959468 >> 172.21.15.144:0/795959468 conn(0x7f50f806aed0 msgr2=0x7f50f806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:53.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.512+0000 7f50ff357700 1 -- 172.21.15.144:0/795959468 shutdown_connections 2023-11-28T11:55:53.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.512+0000 7f50ff357700 1 -- 172.21.15.144:0/795959468 wait complete. 2023-11-28T11:55:53.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.513+0000 7f50ff357700 1 Processor -- start 2023-11-28T11:55:53.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.513+0000 7f50ff357700 1 -- start start 2023-11-28T11:55:53.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4003f80 0x7f50f80b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.521 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 0x7f50f80b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80b5f90 0x7f50f80b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f813fc00 con 0x7f50f813eed0 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f813f640 con 0x7f50e4003f80 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f813f920 con 0x7f50f80b5f90 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 0x7f50f80b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 0x7f50f80b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39052/0 (socket says 172.21.15.144:39052) 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 -- 172.21.15.144:0/1538186742 learned_addr learned my addr 172.21.15.144:0/1538186742 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 -- 172.21.15.144:0/1538186742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80b5f90 msgr2=0x7f50f80b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80b5f90 0x7f50f80b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.522 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 -- 172.21.15.144:0/1538186742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4003f80 msgr2=0x7f50f80b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4003f80 0x7f50f80b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 -- 172.21.15.144:0/1538186742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80b35f0 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50fc8f2700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 0x7f50f80b5a50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f50e4004770 tx=0x7f50f4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4004870 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f40049d0 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f80b3310 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4004d40 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.514+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8140150 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.515+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f8068690 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.516+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f50f4010070 con 0x7f50f813eed0 2023-11-28T11:55:53.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.516+0000 7f50edffb700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50dc0682f0 0x7f50dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:53.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.516+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f50f4086450 con 0x7f50f813eed0 2023-11-28T11:55:53.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.516+0000 7f50fd0f3700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50dc0682f0 0x7f50dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:53.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.517+0000 7f50fd0f3700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50dc0682f0 0x7f50dc06a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f50f80b6d90 tx=0x7f50e8006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:53.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.520+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f50f4052ce0 con 0x7f50f813eed0 2023-11-28T11:55:53.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.743+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f50f815df30 con 0x7f50f813eed0 2023-11-28T11:55:53.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.743+0000 7f50edffb700 1 -- 172.21.15.144:0/1538186742 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f50f4056300 con 0x7f50f813eed0 2023-11-28T11:55:53.745 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:53.745 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50dc0682f0 msgr2=0x7f50dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50dc0682f0 0x7f50dc06a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f50f80b6d90 tx=0x7f50e8006c60 comp rx=0 tx=0).stop 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 msgr2=0x7f50f80b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 0x7f50f80b5a50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f50e4004770 tx=0x7f50f4001a00 comp rx=0 tx=0).stop 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 shutdown_connections 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f50dc0682f0 0x7f50dc06a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50f80b5f90 0x7f50f80b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50e4003f80 0x7f50f80b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 --2- 172.21.15.144:0/1538186742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f813eed0 0x7f50f80b5a50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 >> 172.21.15.144:0/1538186742 conn(0x7f50f806aed0 msgr2=0x7f50f8159f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 shutdown_connections 2023-11-28T11:55:53.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:53.746+0000 7f50ff357700 1 -- 172.21.15.144:0/1538186742 wait complete. 2023-11-28T11:55:53.760 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:55:58.761 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:55:58.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.891+0000 7fbe66c9e700 1 Processor -- start 2023-11-28T11:55:58.893 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.891+0000 7fbe66c9e700 1 -- start start 2023-11-28T11:55:58.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe66c9e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbe6006dee0 0x7fbe6006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:58.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe66c9e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe60160540 0x7fbe60160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe66c9e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6015d060 con 0x7fbe60071040 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe66c9e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6015d340 con 0x7fbe60160540 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe66c9e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6015d620 con 0x7fbe6006dee0 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe6523b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe60160540 0x7fbe60160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe64a3a700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbe60071040 0x7fbe60071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55770/0 (socket says 172.21.15.144:55770) 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe64a3a700 1 -- 172.21.15.144:0/3353740788 learned_addr learned my addr 172.21.15.144:0/3353740788 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.892+0000 7fbe5ffff700 1 --2- 172.21.15.144:0/3353740788 >> v2:172.21.15.177:3300/0 conn(0x7fbe6006dee0 0x7fbe6006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe6523b700 1 -- 172.21.15.144:0/3353740788 >> v2:172.21.15.177:3300/0 conn(0x7fbe6006dee0 msgr2=0x7fbe6006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe6523b700 1 --2- 172.21.15.144:0/3353740788 >> v2:172.21.15.177:3300/0 conn(0x7fbe6006dee0 0x7fbe6006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe6523b700 1 -- 172.21.15.144:0/3353740788 >> v1:172.21.15.144:6789/0 conn(0x7fbe60071040 legacy=0x7fbe60071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe6523b700 1 -- 172.21.15.144:0/3353740788 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6015d900 con 0x7fbe60160540 2023-11-28T11:55:58.896 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe5f7fe700 1 -- 172.21.15.144:0/3353740788 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864497047 0 0) 0x7fbe6015d060 con 0x7fbe60071040 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe5ffff700 1 --2- 172.21.15.144:0/3353740788 >> v2:172.21.15.177:3300/0 conn(0x7fbe6006dee0 0x7fbe6006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe6523b700 1 --2- 172.21.15.144:0/3353740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60160540 0x7fbe60160980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbe6006fb00 tx=0x7fbe5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321cf80b4f50182 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe5f7fe700 1 -- 172.21.15.144:0/3353740788 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800e070 con 0x7fbe60160540 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe5f7fe700 1 -- 172.21.15.144:0/3353740788 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5800bbd0 con 0x7fbe60160540 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe5f7fe700 1 -- 172.21.15.144:0/3353740788 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58012430 con 0x7fbe60160540 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3353740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60160540 msgr2=0x7fbe60160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3353740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60160540 0x7fbe60160980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbe6006fb00 tx=0x7fbe5800b330 comp rx=0 tx=0).stop 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3353740788 shutdown_connections 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3353740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60160540 0x7fbe60160980 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3353740788 >> v2:172.21.15.177:3300/0 conn(0x7fbe6006dee0 0x7fbe6006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3353740788 >> 172.21.15.144:0/3353740788 conn(0x7fbe600d0000 msgr2=0x7fbe600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:58.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3353740788 shutdown_connections 2023-11-28T11:55:58.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.893+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3353740788 wait complete. 2023-11-28T11:55:58.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.895+0000 7fbe66c9e700 1 Processor -- start 2023-11-28T11:55:58.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.895+0000 7fbe66c9e700 1 -- start start 2023-11-28T11:55:58.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.895+0000 7fbe66c9e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe6006dee0 0x7fbe600cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 0x7fbe60149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe60160540 0x7fbe6014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6014a920 con 0x7fbe60071040 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600c5ad0 con 0x7fbe6006dee0 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600c5db0 con 0x7fbe60160540 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 0x7fbe60149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:58.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 0x7fbe60149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39064/0 (socket says 172.21.15.144:39064) 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 -- 172.21.15.144:0/3055156405 learned_addr learned my addr 172.21.15.144:0/3055156405 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe64a3a700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe6006dee0 0x7fbe600cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe6523b700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe60160540 0x7fbe6014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 -- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe60160540 msgr2=0x7fbe6014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe60160540 0x7fbe6014a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 -- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe6006dee0 msgr2=0x7fbe600cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe6006dee0 0x7fbe600cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 -- 172.21.15.144:0/3055156405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58009a70 con 0x7fbe60071040 2023-11-28T11:55:58.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5ffff700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 0x7fbe60149ea0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e810 tx=0x7fbe4c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00c020 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c004250 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6015ccd0 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0036a0 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.896+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6006e870 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.897+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6006a270 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.898+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe4c003800 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.898+0000 7fbe5d7fa700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe440682f0 0x7fbe4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.898+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbe4c085b30 con 0x7fbe60071040 2023-11-28T11:55:58.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.898+0000 7fbe64a3a700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe440682f0 0x7fbe4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:55:58.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.898+0000 7fbe64a3a700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe440682f0 0x7fbe4406a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe54000f60 tx=0x7fbe54007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:55:58.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:58.902+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe4c052270 con 0x7fbe60071040 2023-11-28T11:55:59.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.124+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbe600720a0 con 0x7fbe60071040 2023-11-28T11:55:59.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.124+0000 7fbe5d7fa700 1 -- 172.21.15.144:0/3055156405 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fbe4c00f040 con 0x7fbe60071040 2023-11-28T11:55:59.126 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:55:59.126 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe440682f0 msgr2=0x7fbe4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe440682f0 0x7fbe4406a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe54000f60 tx=0x7fbe54007c90 comp rx=0 tx=0).stop 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 msgr2=0x7fbe60149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 0x7fbe60149ea0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e810 tx=0x7fbe4c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 shutdown_connections 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbe440682f0 0x7fbe4406a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbe60160540 0x7fbe6014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe6006dee0 0x7fbe600cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 --2- 172.21.15.144:0/3055156405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe60071040 0x7fbe60149ea0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:55:59.128 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 >> 172.21.15.144:0/3055156405 conn(0x7fbe600d0000 msgr2=0x7fbe6006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:55:59.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 shutdown_connections 2023-11-28T11:55:59.129 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:55:59.126+0000 7fbe66c9e700 1 -- 172.21.15.144:0/3055156405 wait complete. 2023-11-28T11:55:59.141 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:56:04.141 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:56:04.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.272+0000 7f671e1fd700 1 Processor -- start 2023-11-28T11:56:04.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.272+0000 7f671e1fd700 1 -- start start 2023-11-28T11:56:04.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f671e1fd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6718160b10 0x7f67180cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:04.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f671e1fd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f671806b060 0x7f671806b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:04.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f671e1fd700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67180c9e10 con 0x7f6718160740 2023-11-28T11:56:04.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f671e1fd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67180ca720 con 0x7f6718160b10 2023-11-28T11:56:04.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f671e1fd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671806efc0 con 0x7f671806b060 2023-11-28T11:56:04.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f67177fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6718160740 0x7f67180cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55784/0 (socket says 172.21.15.144:55784) 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f67177fe700 1 -- 172.21.15.144:0/3195556694 learned_addr learned my addr 172.21.15.144:0/3195556694 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f6716ffd700 1 --2- 172.21.15.144:0/3195556694 >> v2:172.21.15.144:3300/0 conn(0x7f6718160b10 0x7f67180cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f67167fc700 1 -- 172.21.15.144:0/3195556694 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056930237 0 0) 0x7f67180c9e10 con 0x7f6718160740 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f67167fc700 1 -- 172.21.15.144:0/3195556694 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67000035b0 con 0x7f6718160740 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f6717fff700 1 --2- 172.21.15.144:0/3195556694 >> v2:172.21.15.177:3300/0 conn(0x7f671806b060 0x7f671806b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f6716ffd700 1 -- 172.21.15.144:0/3195556694 >> v2:172.21.15.177:3300/0 conn(0x7f671806b060 msgr2=0x7f671806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f6716ffd700 1 --2- 172.21.15.144:0/3195556694 >> v2:172.21.15.177:3300/0 conn(0x7f671806b060 0x7f671806b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f6716ffd700 1 -- 172.21.15.144:0/3195556694 >> v1:172.21.15.144:6789/0 conn(0x7f6718160740 legacy=0x7f67180cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.273+0000 7f6716ffd700 1 -- 172.21.15.144:0/3195556694 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180ca0d0 con 0x7f6718160b10 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f6716ffd700 1 --2- 172.21.15.144:0/3195556694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160b10 0x7f67180cd390 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f671806c470 tx=0x7f670c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877727542193b0c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f67167fc700 1 -- 172.21.15.144:0/3195556694 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c011830 con 0x7f6718160b10 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f67167fc700 1 -- 172.21.15.144:0/3195556694 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c011990 con 0x7f6718160b10 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f67167fc700 1 -- 172.21.15.144:0/3195556694 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c011d00 con 0x7f6718160b10 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 -- 172.21.15.144:0/3195556694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160b10 msgr2=0x7f67180cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3195556694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160b10 0x7f67180cd390 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f671806c470 tx=0x7f670c009b20 comp rx=0 tx=0).stop 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 -- 172.21.15.144:0/3195556694 shutdown_connections 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3195556694 >> v2:172.21.15.177:3300/0 conn(0x7f671806b060 0x7f671806b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3195556694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160b10 0x7f67180cd390 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 -- 172.21.15.144:0/3195556694 >> 172.21.15.144:0/3195556694 conn(0x7f67180b12b0 msgr2=0x7f67180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:04.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 -- 172.21.15.144:0/3195556694 shutdown_connections 2023-11-28T11:56:04.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.274+0000 7f671e1fd700 1 -- 172.21.15.144:0/3195556694 wait complete. 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.275+0000 7f671e1fd700 1 Processor -- start 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 -- start start 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671806b060 0x7f67180b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 0x7f67180c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6718160b10 0x7f671815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671814f290 con 0x7f6718160740 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180c3e60 con 0x7f6718160b10 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.276+0000 7f671e1fd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671806dd00 con 0x7f671806b060 2023-11-28T11:56:04.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 0x7f67180c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 0x7f67180c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39080/0 (socket says 172.21.15.144:39080) 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 -- 172.21.15.144:0/3927174795 learned_addr learned my addr 172.21.15.144:0/3927174795 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6717fff700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6718160b10 0x7f671815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f67177fe700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671806b060 0x7f67180b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 -- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671806b060 msgr2=0x7f67180b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671806b060 0x7f67180b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 -- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6718160b10 msgr2=0x7f671815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6718160b10 0x7f671815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 -- 172.21.15.144:0/3927174795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0094a0 con 0x7f6718160740 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6717fff700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6718160b10 0x7f671815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f6716ffd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 0x7f67180c3920 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f670c009b60 tx=0x7f670c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0036a0 con 0x7f6718160740 2023-11-28T11:56:04.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c01f070 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67180cd8d0 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c003a40 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.277+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67180ca3b0 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.278+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6718160f60 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.279+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f670c00f050 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.279+0000 7f66fffff700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f67040682f0 0x7f670406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.279+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f670c021070 con 0x7f6718160740 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.279+0000 7f67177fe700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f67040682f0 0x7f670406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.279+0000 7f67177fe700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f67040682f0 0x7f670406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6708005910 tx=0x7f6708009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:04.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.283+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f670c053e80 con 0x7f6718160740 2023-11-28T11:56:04.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.509+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6718072720 con 0x7f6718160740 2023-11-28T11:56:04.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.509+0000 7f66fffff700 1 -- 172.21.15.144:0/3927174795 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f670c0574a0 con 0x7f6718160740 2023-11-28T11:56:04.512 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:04.512 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:56:04.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f67040682f0 msgr2=0x7f670406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:04.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f67040682f0 0x7f670406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6708005910 tx=0x7f6708009040 comp rx=0 tx=0).stop 2023-11-28T11:56:04.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 msgr2=0x7f67180c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 0x7f67180c3920 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f670c009b60 tx=0x7f670c004750 comp rx=0 tx=0).stop 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 shutdown_connections 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f67040682f0 0x7f670406a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6718160b10 0x7f671815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f671806b060 0x7f67180b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 --2- 172.21.15.144:0/3927174795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6718160740 0x7f67180c3920 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 >> 172.21.15.144:0/3927174795 conn(0x7f67180b12b0 msgr2=0x7f671815e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.511+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 shutdown_connections 2023-11-28T11:56:04.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:04.512+0000 7f671e1fd700 1 -- 172.21.15.144:0/3927174795 wait complete. 2023-11-28T11:56:04.526 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:56:09.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:56:09.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.659+0000 7f1d2f9b4700 1 Processor -- start 2023-11-28T11:56:09.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.659+0000 7f1d2f9b4700 1 -- start start 2023-11-28T11:56:09.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2f9b4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d28154e70 0x7f1d280cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2f9b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 0x7f1d28071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2f9b4700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d280719e0 con 0x7f1d28071580 2023-11-28T11:56:09.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2f9b4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d28155240 con 0x7f1d28154e70 2023-11-28T11:56:09.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2f9b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2815b100 con 0x7f1d280cf360 2023-11-28T11:56:09.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2df51700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1d28071580 0x7f1d2815e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55800/0 (socket says 172.21.15.144:55800) 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2df51700 1 -- 172.21.15.144:0/3850167903 learned_addr learned my addr 172.21.15.144:0/3850167903 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.660+0000 7f1d2d750700 1 --2- 172.21.15.144:0/3850167903 >> v2:172.21.15.177:3300/0 conn(0x7f1d28154e70 0x7f1d280cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 0x7f1d28071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343539136 0 0) 0x7f1d280719e0 con 0x7f1d28071580 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d140035b0 con 0x7f1d28071580 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303353570 0 0) 0x7f1d140035b0 con 0x7f1d28071580 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 msgr2=0x7f1d28071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 0x7f1d28071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 >> v2:172.21.15.177:3300/0 conn(0x7f1d28154e70 msgr2=0x7f1d280cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:09.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 --2- 172.21.15.144:0/3850167903 >> v2:172.21.15.177:3300/0 conn(0x7f1d28154e70 0x7f1d280cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2815b3e0 con 0x7f1d28071580 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f1d200019f0 con 0x7f1d28071580 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 >> v1:172.21.15.144:6789/0 conn(0x7f1d28071580 legacy=0x7f1d2815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 0x7f1d14003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d14003f80 0x7f1d14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d14004d90 0x7f1d14005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d28155240 con 0x7f1d14003f80 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2815b100 con 0x7f1d14004d90 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1400ba40 con 0x7f1d280cf360 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d14003f80 0x7f1d14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2d750700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 0x7f1d14003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2df51700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d14004d90 0x7f1d14005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 -- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 msgr2=0x7f1d14003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d280cf360 0x7f1d14003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 -- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d14004d90 msgr2=0x7f1d14005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d14004d90 0x7f1d14005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.661+0000 7f1d2cf4f700 1 -- 172.21.15.144:0/3850167903 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d140035b0 con 0x7f1d14003f80 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2cf4f700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d14003f80 0x7f1d14004850 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009b60 tx=0x7f1d14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4eb5497f88d9b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400e660 con 0x7f1d14003f80 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2400e7c0 con 0x7f1d14003f80 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d1bfff700 1 -- 172.21.15.144:0/3850167903 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400eb30 con 0x7f1d14003f80 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/3850167903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d14003f80 msgr2=0x7f1d14004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:09.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/3850167903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d14003f80 0x7f1d14004850 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009b60 tx=0x7f1d14005880 comp rx=0 tx=0).stop 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2df51700 1 -- 172.21.15.144:0/3850167903 reap_dead start 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/3850167903 shutdown_connections 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/3850167903 >> 172.21.15.144:0/3850167903 conn(0x7f1d280d0000 msgr2=0x7f1d280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/3850167903 shutdown_connections 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.662+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/3850167903 wait complete. 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 Processor -- start 2023-11-28T11:56:09.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 -- start start 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d14003f80 0x7f1d28153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d280ade40 0x7f1d280ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 0x7f1d280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280ae7f0 con 0x7f1d280b4960 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280b5690 con 0x7f1d280ade40 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2f9b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280b2fd0 con 0x7f1d14003f80 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2df51700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 0x7f1d280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2df51700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 0x7f1d280b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39100/0 (socket says 172.21.15.144:39100) 2023-11-28T11:56:09.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2df51700 1 -- 172.21.15.144:0/1255149378 learned_addr learned my addr 172.21.15.144:0/1255149378 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.664+0000 7f1d2d750700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d14003f80 0x7f1d28153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2cf4f700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d280ade40 0x7f1d280ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2df51700 1 -- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d14003f80 msgr2=0x7f1d28153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2df51700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d14003f80 0x7f1d28153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2df51700 1 -- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d280ade40 msgr2=0x7f1d280ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2df51700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d280ade40 0x7f1d280ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2df51700 1 -- 172.21.15.144:0/1255149378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2815b9a0 con 0x7f1d280b4960 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2df51700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 0x7f1d280b4dd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1d20001330 tx=0x7f1d20007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:09.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20001f00 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2000ebc0 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20013420 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2815b6c0 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.665+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d280af520 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.666+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d28071e00 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.667+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1d20013580 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.667+0000 7f1d19ffb700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d0c0682f0 0x7f1d0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.667+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1d20086360 con 0x7f1d280b4960 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.667+0000 7f1d2d750700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d0c0682f0 0x7f1d0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:09.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.667+0000 7f1d2d750700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d0c0682f0 0x7f1d0c06a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009ce0 tx=0x7f1d1c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:09.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.671+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1d20052b30 con 0x7f1d280b4960 2023-11-28T11:56:09.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.896+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1d280ce4d0 con 0x7f1d280b4960 2023-11-28T11:56:09.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.896+0000 7f1d19ffb700 1 -- 172.21.15.144:0/1255149378 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1d20056150 con 0x7f1d280b4960 2023-11-28T11:56:09.897 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:09.897 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:56:09.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d0c0682f0 msgr2=0x7f1d0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:09.899 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d0c0682f0 0x7f1d0c06a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009ce0 tx=0x7f1d1c009210 comp rx=0 tx=0).stop 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 msgr2=0x7f1d280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 0x7f1d280b4dd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1d20001330 tx=0x7f1d20007c90 comp rx=0 tx=0).stop 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 shutdown_connections 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d280b4960 0x7f1d280b4dd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d0c0682f0 0x7f1d0c06a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d14003f80 0x7f1d28153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 --2- 172.21.15.144:0/1255149378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d280ade40 0x7f1d280ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 >> 172.21.15.144:0/1255149378 conn(0x7f1d280d0000 msgr2=0x7f1d2806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.898+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 shutdown_connections 2023-11-28T11:56:09.900 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:09.899+0000 7f1d2f9b4700 1 -- 172.21.15.144:0/1255149378 wait complete. 2023-11-28T11:56:09.911 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:56:14.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:56:15.047 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.046+0000 7f74d593b700 1 Processor -- start 2023-11-28T11:56:15.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.046+0000 7f74d593b700 1 -- start start 2023-11-28T11:56:15.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74d593b700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f74d006dee0 0x7f74d006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:15.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74d593b700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d015d060 con 0x7f74d0071040 2023-11-28T11:56:15.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74d593b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d015d340 con 0x7f74d0160540 2023-11-28T11:56:15.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74d593b700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d015d620 con 0x7f74d006dee0 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74ce7fc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f74d006dee0 0x7f74d006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74ceffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f74d0071040 0x7f74d0071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55818/0 (socket says 172.21.15.144:55818) 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.047+0000 7f74ceffd700 1 -- 172.21.15.144:0/478380044 learned_addr learned my addr 172.21.15.144:0/478380044 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74ce7fc700 1 -- 172.21.15.144:0/478380044 >> v1:172.21.15.177:6789/0 conn(0x7f74d0160540 legacy=0x7f74d0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74ce7fc700 1 -- 172.21.15.144:0/478380044 >> v1:172.21.15.144:6789/0 conn(0x7f74d0071040 legacy=0x7f74d0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74ce7fc700 1 -- 172.21.15.144:0/478380044 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d015d900 con 0x7f74d006dee0 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74cdffb700 1 -- 172.21.15.144:0/478380044 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484497111 0 0) 0x7f74d015d060 con 0x7f74d0071040 2023-11-28T11:56:15.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74ce7fc700 1 --2- 172.21.15.144:0/478380044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d006dee0 0x7f74d006e330 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009840 tx=0x7f74c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e566a0b1d3e90540 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74cdffb700 1 -- 172.21.15.144:0/478380044 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004230 con 0x7f74d006dee0 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74cdffb700 1 -- 172.21.15.144:0/478380044 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0004390 con 0x7f74d006dee0 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74cdffb700 1 -- 172.21.15.144:0/478380044 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00036a0 con 0x7f74d006dee0 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74d593b700 1 -- 172.21.15.144:0/478380044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d006dee0 msgr2=0x7f74d006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.048+0000 7f74d593b700 1 --2- 172.21.15.144:0/478380044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d006dee0 0x7f74d006e330 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009840 tx=0x7f74c0009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.049+0000 7f74d593b700 1 -- 172.21.15.144:0/478380044 shutdown_connections 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.049+0000 7f74d593b700 1 --2- 172.21.15.144:0/478380044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d006dee0 0x7f74d006e330 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.049+0000 7f74d593b700 1 -- 172.21.15.144:0/478380044 >> 172.21.15.144:0/478380044 conn(0x7f74d00d0000 msgr2=0x7f74d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.049+0000 7f74d593b700 1 -- 172.21.15.144:0/478380044 shutdown_connections 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.049+0000 7f74d593b700 1 -- 172.21.15.144:0/478380044 wait complete. 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.050+0000 7f74d593b700 1 Processor -- start 2023-11-28T11:56:15.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.050+0000 7f74d593b700 1 -- start start 2023-11-28T11:56:15.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74d006dee0 0x7f74d01f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:15.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 0x7f74d00b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:15.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74d0160540 0x7f74d0149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:15.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d00b5c90 con 0x7f74d0071040 2023-11-28T11:56:15.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01e5650 con 0x7f74d006dee0 2023-11-28T11:56:15.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01e5910 con 0x7f74d0160540 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 0x7f74d00b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 0x7f74d00b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39114/0 (socket says 172.21.15.144:39114) 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 -- 172.21.15.144:0/1502851292 learned_addr learned my addr 172.21.15.144:0/1502851292 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ceffd700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74d006dee0 0x7f74d01f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74cf7fe700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74d0160540 0x7f74d0149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 -- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74d0160540 msgr2=0x7f74d0149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74d0160540 0x7f74d0149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 -- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74d006dee0 msgr2=0x7f74d01f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74d006dee0 0x7f74d01f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 -- 172.21.15.144:0/1502851292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00094a0 con 0x7f74d0071040 2023-11-28T11:56:15.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74ce7fc700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 0x7f74d00b5750 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f74c00090b0 tx=0x7f74c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0011830 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0011990 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0011d40 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d015ccd0 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.051+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d006e870 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.052+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d006a270 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.053+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74c0017020 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.053+0000 7f74c7fff700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f74b0068320 0x7f74b006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.053+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f74c00873f0 con 0x7f74d0071040 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.053+0000 7f74ceffd700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f74b0068320 0x7f74b006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:15.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.054+0000 7f74ceffd700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f74b0068320 0x7f74b006a7d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f74b8000f60 tx=0x7f74b8007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:15.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.057+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f74c0053bc0 con 0x7f74d0071040 2023-11-28T11:56:15.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.279+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f74d00720a0 con 0x7f74d0071040 2023-11-28T11:56:15.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.280+0000 7f74c7fff700 1 -- 172.21.15.144:0/1502851292 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f74c00571e0 con 0x7f74d0071040 2023-11-28T11:56:15.281 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:15.281 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:56:15.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f74b0068320 msgr2=0x7f74b006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:15.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f74b0068320 0x7f74b006a7d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f74b8000f60 tx=0x7f74b8007c90 comp rx=0 tx=0).stop 2023-11-28T11:56:15.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 msgr2=0x7f74d00b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 0x7f74d00b5750 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f74c00090b0 tx=0x7f74c00047c0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 shutdown_connections 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f74b0068320 0x7f74b006a7d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74d0160540 0x7f74d0149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74d006dee0 0x7f74d01f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 --2- 172.21.15.144:0/1502851292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74d0071040 0x7f74d00b5750 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 >> 172.21.15.144:0/1502851292 conn(0x7f74d00d0000 msgr2=0x7f74d00cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 shutdown_connections 2023-11-28T11:56:15.284 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:15.282+0000 7f74d593b700 1 -- 172.21.15.144:0/1502851292 wait complete. 2023-11-28T11:56:15.295 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2023-11-28T11:56:20.296 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:56:20.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.428+0000 7f2c2f33e700 1 Processor -- start 2023-11-28T11:56:20.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.429+0000 7f2c2f33e700 1 -- start start 2023-11-28T11:56:20.438 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.429+0000 7f2c2f33e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c28071040 0x7f2c28071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:20.438 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2f33e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28160540 0x7f2c281609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:20.438 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2f33e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c28071950 con 0x7f2c2806dee0 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2f33e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2815d060 con 0x7f2c28071040 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2f33e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2815d340 con 0x7f2c28160540 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c28071040 0x7f2c28071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c28071040 0x7f2c28071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39120/0 (socket says 172.21.15.144:39120) 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 -- 172.21.15.144:0/2855774059 learned_addr learned my addr 172.21.15.144:0/2855774059 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d8db700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28160540 0x7f2c281609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 -- 172.21.15.144:0/2855774059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28160540 msgr2=0x7f2c281609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28160540 0x7f2c281609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 -- 172.21.15.144:0/2855774059 >> v1:172.21.15.177:6789/0 conn(0x7f2c2806dee0 legacy=0x7f2c2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 -- 172.21.15.144:0/2855774059 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2815d620 con 0x7f2c28071040 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d8db700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28160540 0x7f2c281609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c2d0da700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28071040 0x7f2c28071410 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009e50 tx=0x7f2c1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b6dca205c5725c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c27fff700 1 -- 172.21.15.144:0/2855774059 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800e070 con 0x7f2c28071040 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c27fff700 1 -- 172.21.15.144:0/2855774059 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c18003a20 con 0x7f2c28071040 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.430+0000 7f2c27fff700 1 -- 172.21.15.144:0/2855774059 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18003dd0 con 0x7f2c28071040 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 -- 172.21.15.144:0/2855774059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28071040 msgr2=0x7f2c28071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28071040 0x7f2c28071410 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009e50 tx=0x7f2c1800b330 comp rx=0 tx=0).stop 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 -- 172.21.15.144:0/2855774059 shutdown_connections 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28160540 0x7f2c281609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/2855774059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28071040 0x7f2c28071410 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 -- 172.21.15.144:0/2855774059 >> 172.21.15.144:0/2855774059 conn(0x7f2c280d0000 msgr2=0x7f2c280d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 -- 172.21.15.144:0/2855774059 shutdown_connections 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.431+0000 7f2c2f33e700 1 -- 172.21.15.144:0/2855774059 wait complete. 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.432+0000 7f2c2f33e700 1 Processor -- start 2023-11-28T11:56:20.440 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.432+0000 7f2c2f33e700 1 -- start start 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2f33e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c2806dee0 0x7f2c2814d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2f33e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28071040 0x7f2c280ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2f33e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 0x7f2c280ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2f33e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280b5770 con 0x7f2c28160540 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2f33e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280c6960 con 0x7f2c2806dee0 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2f33e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c280b4ba0 con 0x7f2c28071040 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 0x7f2c280ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 0x7f2c280ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39130/0 (socket says 172.21.15.144:39130) 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 -- 172.21.15.144:0/1358998524 learned_addr learned my addr 172.21.15.144:0/1358998524 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2c8d9700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28071040 0x7f2c280ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d0da700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c2806dee0 0x7f2c2814d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:20.441 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 -- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28071040 msgr2=0x7f2c280ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28071040 0x7f2c280ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 -- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c2806dee0 msgr2=0x7f2c2814d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c2806dee0 0x7f2c2814d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 -- 172.21.15.144:0/1358998524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18009b30 con 0x7f2c28160540 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d0da700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c2806dee0 0x7f2c2814d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2c8d9700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28071040 0x7f2c280ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.433+0000 7f2c2d8db700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 0x7f2c280ae380 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c7f0 tx=0x7f2c2000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.434+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011800 con 0x7f2c28160540 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.434+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20011960 con 0x7f2c28160540 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.434+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2815ccd0 con 0x7f2c28160540 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.434+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011d10 con 0x7f2c28160540 2023-11-28T11:56:20.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.434+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2815d900 con 0x7f2c28160540 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.435+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2806a270 con 0x7f2c28160540 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.435+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c20021070 con 0x7f2c28160540 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.436+0000 7f2c25ffb700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c140682f0 0x7f2c1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.436+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2c20085c20 con 0x7f2c28160540 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.436+0000 7f2c2d0da700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c140682f0 0x7f2c1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.436+0000 7f2c2d0da700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c140682f0 0x7f2c1406a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2c18015040 tx=0x7f2c18009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:20.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.440+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c20052310 con 0x7f2c28160540 2023-11-28T11:56:20.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.665+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2c280b48f0 con 0x7f2c28160540 2023-11-28T11:56:20.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.665+0000 7f2c25ffb700 1 -- 172.21.15.144:0/1358998524 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f2c20055930 con 0x7f2c28160540 2023-11-28T11:56:20.666 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:20.666 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:56:20.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c140682f0 msgr2=0x7f2c1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:20.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c140682f0 0x7f2c1406a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2c18015040 tx=0x7f2c18009000 comp rx=0 tx=0).stop 2023-11-28T11:56:20.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 msgr2=0x7f2c280ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 0x7f2c280ae380 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c7f0 tx=0x7f2c2000cbb0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 shutdown_connections 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c28160540 0x7f2c280ae380 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2c140682f0 0x7f2c1406a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c2806dee0 0x7f2c2814d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 --2- 172.21.15.144:0/1358998524 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c28071040 0x7f2c280ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:20.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 >> 172.21.15.144:0/1358998524 conn(0x7f2c280d0000 msgr2=0x7f2c280cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:20.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 shutdown_connections 2023-11-28T11:56:20.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:20.667+0000 7f2c2f33e700 1 -- 172.21.15.144:0/1358998524 wait complete. 2023-11-28T11:56:20.680 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:56:20.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:56:20.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:56:20.762 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:56:20.762 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:56:20.762 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:20.782 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:56:20.782 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:20.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:56:21.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.000+0000 7f1b4b59e700 1 Processor -- start 2023-11-28T11:56:21.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.000+0000 7f1b4b59e700 1 -- start start 2023-11-28T11:56:21.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4b59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4b59e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1b4c0ac8b0 0x7f1b4c0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4b59e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4c0b5050 con 0x7f1b4c06a270 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4b59e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c071730 con 0x7f1b4c0ac8b0 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4b59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cf830 con 0x7f1b4c0b3a70 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1b4c0ac8b0 0x7f1b4c0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4ad9d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1b4c0ac8b0 0x7f1b4c0cff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39138/0 (socket says 172.21.15.144:39138) 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.001+0000 7f1b4ad9d700 1 -- 172.21.15.144:0/2647833025 learned_addr learned my addr 172.21.15.144:0/2647833025 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b49d9b700 1 --2- 172.21.15.144:0/2647833025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4ad9d700 1 -- 172.21.15.144:0/2647833025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0b3a70 msgr2=0x7f1b4c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4ad9d700 1 --2- 172.21.15.144:0/2647833025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4ad9d700 1 -- 172.21.15.144:0/2647833025 >> v1:172.21.15.177:6789/0 conn(0x7f1b4c06a270 legacy=0x7f1b4c06a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4ad9d700 1 -- 172.21.15.144:0/2647833025 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0b3f10 con 0x7f1b4c0ac8b0 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4ad9d700 1 --2- 172.21.15.144:0/2647833025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0cff60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009e50 tx=0x7f1b4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e007a3ecab3f06a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4959a700 1 -- 172.21.15.144:0/2647833025 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400e070 con 0x7f1b4c0ac8b0 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4959a700 1 -- 172.21.15.144:0/2647833025 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4400b9c0 con 0x7f1b4c0ac8b0 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.002+0000 7f1b4959a700 1 -- 172.21.15.144:0/2647833025 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400bd70 con 0x7f1b4c0ac8b0 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 -- 172.21.15.144:0/2647833025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c0ac8b0 msgr2=0x7f1b4c0cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/2647833025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0cff60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009e50 tx=0x7f1b4400b330 comp rx=0 tx=0).stop 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 -- 172.21.15.144:0/2647833025 shutdown_connections 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/2647833025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0cff60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/2647833025 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 -- 172.21.15.144:0/2647833025 >> 172.21.15.144:0/2647833025 conn(0x7f1b4c06b380 msgr2=0x7f1b4c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:21.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 -- 172.21.15.144:0/2647833025 shutdown_connections 2023-11-28T11:56:21.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.003+0000 7f1b4b59e700 1 -- 172.21.15.144:0/2647833025 wait complete. 2023-11-28T11:56:21.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.004+0000 7f1b4b59e700 1 Processor -- start 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.004+0000 7f1b4b59e700 1 -- start start 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 0x7f1b4c1447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cc660 con 0x7f1b4c06a270 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15e480 con 0x7f1b4c0b3a70 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15e760 con 0x7f1b4c0ac8b0 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 0x7f1b4c1447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 0x7f1b4c1447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39140/0 (socket says 172.21.15.144:39140) 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 -- 172.21.15.144:0/3044533206 learned_addr learned my addr 172.21.15.144:0/3044533206 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:21.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4ad9d700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b49d9b700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 -- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0ac8b0 msgr2=0x7f1b4c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 -- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b4c0b3a70 msgr2=0x7f1b4c0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 -- 172.21.15.144:0/3044533206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009b30 con 0x7f1b4c06a270 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b49d9b700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4a59c700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 0x7f1b4c1447e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c4e0 tx=0x7f1b3c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0052d0 con 0x7f1b4c06a270 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c005430 con 0x7f1b4c06a270 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c071c20 con 0x7f1b4c06a270 2023-11-28T11:56:21.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c0038c0 con 0x7f1b4c06a270 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.005+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c071940 con 0x7f1b4c06a270 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.007+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c0cfa40 con 0x7f1b4c06a270 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.007+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b3c019030 con 0x7f1b4c06a270 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.007+0000 7f1b42ffd700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.007+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1b3c085080 con 0x7f1b4c06a270 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.007+0000 7f1b49d9b700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.008+0000 7f1b49d9b700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c06bc00 tx=0x7f1b34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.011+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b3c051850 con 0x7f1b4c06a270 2023-11-28T11:56:21.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.193+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b4c0cc150 con 0x7f1b4c06a270 2023-11-28T11:56:21.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.194+0000 7f1b42ffd700 1 -- 172.21.15.144:0/3044533206 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18151 (secure 0 0 0) 0x7f1b4c0cc150 con 0x7f1b4c06a270 2023-11-28T11:56:21.195 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:21.195 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":62,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:54:06.735804+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1b2c0682f0 msgr2=0x7f1b2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c06bc00 tx=0x7f1b34008040 comp rx=0 tx=0).stop 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 msgr2=0x7f1b4c1447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 0x7f1b4c1447e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c4e0 tx=0x7f1b3c00c8a0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 shutdown_connections 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c06a270 0x7f1b4c1447e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b4c0b3a70 0x7f1b4c0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 --2- 172.21.15.144:0/3044533206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b4c0ac8b0 0x7f1b4c0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 >> 172.21.15.144:0/3044533206 conn(0x7f1b4c06b380 msgr2=0x7f1b4c15b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:21.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 shutdown_connections 2023-11-28T11:56:21.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.196+0000 7f1b4b59e700 1 -- 172.21.15.144:0/3044533206 wait complete. 2023-11-28T11:56:21.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:53:51.132040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:53:52.295120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:56:21.209 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:56:21.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.331+0000 7f89ef63a700 1 Processor -- start 2023-11-28T11:56:21.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.332+0000 7f89ef63a700 1 -- start start 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.332+0000 7f89ef63a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 0x7f89e80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.332+0000 7f89ef63a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e80d2320 0x7f89e80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.332+0000 7f89ef63a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e80d1950 con 0x7f89e815c9b0 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.332+0000 7f89ef63a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e80b5800 con 0x7f89e8161340 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.332+0000 7f89ef63a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80b5300 con 0x7f89e80d2320 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89ecbd5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 0x7f89e80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89edbd7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e80d2320 0x7f89e80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89ecbd5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 0x7f89e80d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:41422/0 (socket says 172.21.15.144:41422) 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89ecbd5700 1 -- 172.21.15.144:0/3780687641 learned_addr learned my addr 172.21.15.144:0/3780687641 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:21.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89edbd7700 1 -- 172.21.15.144:0/3780687641 >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 msgr2=0x7f89e80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89edbd7700 1 --2- 172.21.15.144:0/3780687641 >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 0x7f89e80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89edbd7700 1 -- 172.21.15.144:0/3780687641 >> v1:172.21.15.177:6789/0 conn(0x7f89e815c9b0 legacy=0x7f89e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89edbd7700 1 -- 172.21.15.144:0/3780687641 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80b5520 con 0x7f89e80d2320 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89ecbd5700 1 --2- 172.21.15.144:0/3780687641 >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 0x7f89e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.333+0000 7f89edbd7700 1 --2- 172.21.15.144:0/3780687641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e80d2320 0x7f89e80cd7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009d90 tx=0x7f89e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e2d062a6365ed6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89dffff700 1 -- 172.21.15.144:0/3780687641 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000e070 con 0x7f89e80d2320 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89dffff700 1 -- 172.21.15.144:0/3780687641 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000bb20 con 0x7f89e80d2320 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89dffff700 1 -- 172.21.15.144:0/3780687641 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0012430 con 0x7f89e80d2320 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 -- 172.21.15.144:0/3780687641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e80d2320 msgr2=0x7f89e80cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 --2- 172.21.15.144:0/3780687641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e80d2320 0x7f89e80cd7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009d90 tx=0x7f89e000b330 comp rx=0 tx=0).stop 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 -- 172.21.15.144:0/3780687641 shutdown_connections 2023-11-28T11:56:21.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 --2- 172.21.15.144:0/3780687641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e80d2320 0x7f89e80cd7c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 --2- 172.21.15.144:0/3780687641 >> v2:172.21.15.177:3300/0 conn(0x7f89e8161340 0x7f89e80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 -- 172.21.15.144:0/3780687641 >> 172.21.15.144:0/3780687641 conn(0x7f89e8072600 msgr2=0x7f89e80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:21.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 -- 172.21.15.144:0/3780687641 shutdown_connections 2023-11-28T11:56:21.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.334+0000 7f89ef63a700 1 -- 172.21.15.144:0/3780687641 wait complete. 2023-11-28T11:56:21.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.335+0000 7f89ef63a700 1 Processor -- start 2023-11-28T11:56:21.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 -- start start 2023-11-28T11:56:21.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89e80d2320 0x7f89e81e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 0x7f89e81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e8161340 0x7f89e81f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81e73d0 con 0x7f89e815c9b0 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81e7670 con 0x7f89e80d2320 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ef63a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81e7950 con 0x7f89e8161340 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ecbd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 0x7f89e81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ecbd5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 0x7f89e81577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39154/0 (socket says 172.21.15.144:39154) 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ecbd5700 1 -- 172.21.15.144:0/1685616977 learned_addr learned my addr 172.21.15.144:0/1685616977 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89edbd7700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e8161340 0x7f89e81f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.336+0000 7f89ed3d6700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89e80d2320 0x7f89e81e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ecbd5700 1 -- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e8161340 msgr2=0x7f89e81f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ecbd5700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e8161340 0x7f89e81f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ecbd5700 1 -- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89e80d2320 msgr2=0x7f89e81e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ecbd5700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89e80d2320 0x7f89e81e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ecbd5700 1 -- 172.21.15.144:0/1685616977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009550 con 0x7f89e815c9b0 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ecbd5700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 0x7f89e81577c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f89e400ebe0 tx=0x7f89e400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400c850 con 0x7f89e815c9b0 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e400c9b0 con 0x7f89e815c9b0 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e400cd40 con 0x7f89e815c9b0 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e0009a70 con 0x7f89e815c9b0 2023-11-28T11:56:21.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.337+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e80d15e0 con 0x7f89e815c9b0 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.338+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e80d1300 con 0x7f89e815c9b0 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.339+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f89e4019030 con 0x7f89e815c9b0 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.339+0000 7f89ddffb700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89cc0682f0 0x7f89cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.339+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f89e4085da0 con 0x7f89e815c9b0 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.339+0000 7f89ed3d6700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89cc0682f0 0x7f89cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.339+0000 7f89ed3d6700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89cc0682f0 0x7f89cc06a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f89d8001800 tx=0x7f89d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.343+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89e4052570 con 0x7f89e815c9b0 2023-11-28T11:56:21.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.524+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89e81e8560 con 0x7f89e815c9b0 2023-11-28T11:56:21.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.525+0000 7f89ddffb700 1 -- 172.21.15.144:0/1685616977 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89e40170a0 con 0x7f89e815c9b0 2023-11-28T11:56:21.525 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89cc0682f0 msgr2=0x7f89cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89cc0682f0 0x7f89cc06a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f89d8001800 tx=0x7f89d8009040 comp rx=0 tx=0).stop 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 msgr2=0x7f89e81577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 0x7f89e81577c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f89e400ebe0 tx=0x7f89e400eef0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 shutdown_connections 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f89cc0682f0 0x7f89cc06a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89e8161340 0x7f89e81f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89e80d2320 0x7f89e81e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 --2- 172.21.15.144:0/1685616977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89e815c9b0 0x7f89e81577c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 >> 172.21.15.144:0/1685616977 conn(0x7f89e8072600 msgr2=0x7f89e80b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 shutdown_connections 2023-11-28T11:56:21.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.527+0000 7f89ef63a700 1 -- 172.21.15.144:0/1685616977 wait complete. 2023-11-28T11:56:21.541 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:56:21.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.664+0000 7f7bf70c3700 1 Processor -- start 2023-11-28T11:56:21.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.664+0000 7f7bf70c3700 1 -- start start 2023-11-28T11:56:21.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf70c3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 0x7f7bf00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf70c3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7bf00cbdf0 0x7f7bf00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf70c3700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf00cc5f0 con 0x7f7bf015cef0 2023-11-28T11:56:21.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf70c3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf015d470 con 0x7f7bf00cbdf0 2023-11-28T11:56:21.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf70c3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015d750 con 0x7f7bf006ba90 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf4e5f700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7bf015cef0 0x7f7bf006b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55872/0 (socket says 172.21.15.144:55872) 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf4e5f700 1 -- 172.21.15.144:0/4075424841 learned_addr learned my addr 172.21.15.144:0/4075424841 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943238758 0 0) 0x7f7bf00cc5f0 con 0x7f7bf015cef0 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be00035b0 con 0x7f7bf015cef0 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7bf5660700 1 --2- 172.21.15.144:0/4075424841 >> v2:172.21.15.177:3300/0 conn(0x7f7bf00cbdf0 0x7f7bf00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.665+0000 7f7beffff700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 0x7f7bf00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137687773 0 0) 0x7f7be00035b0 con 0x7f7bf015cef0 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 msgr2=0x7f7bf00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 0x7f7bf00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 >> v2:172.21.15.177:3300/0 conn(0x7f7bf00cbdf0 msgr2=0x7f7bf00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 --2- 172.21.15.144:0/4075424841 >> v2:172.21.15.177:3300/0 conn(0x7f7bf00cbdf0 0x7f7bf00ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf015da30 con 0x7f7bf015cef0 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f7be4001670 con 0x7f7bf015cef0 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 >> v1:172.21.15.144:6789/0 conn(0x7f7bf015cef0 legacy=0x7f7bf006b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 0x7f7be0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7be0003f80 0x7f7be0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0004d90 0x7f7be0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015d470 con 0x7f7be0003f80 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015d750 con 0x7f7be0004d90 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be000ba40 con 0x7f7bf006ba90 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7be0003f80 0x7f7be0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bf4e5f700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 0x7f7be0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bf5660700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0004d90 0x7f7be0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 -- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 msgr2=0x7f7be0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf006ba90 0x7f7be0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 -- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0004d90 msgr2=0x7f7be0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0004d90 0x7f7be0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 -- 172.21.15.144:0/4075424841 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00035b0 con 0x7f7be0003f80 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bf5660700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0004d90 0x7f7be0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7beffff700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7be0003f80 0x7f7be0004850 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009b30 tx=0x7f7be0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49bc9a1ad3957c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00e660 con 0x7f7be0003f80 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc00e7c0 con 0x7f7be0003f80 2023-11-28T11:56:21.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.666+0000 7f7bef7fe700 1 -- 172.21.15.144:0/4075424841 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00eb30 con 0x7f7be0003f80 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf70c3700 1 -- 172.21.15.144:0/4075424841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7be0003f80 msgr2=0x7f7be0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/4075424841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7be0003f80 0x7f7be0004850 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009b30 tx=0x7f7be0005880 comp rx=0 tx=0).stop 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf5660700 1 -- 172.21.15.144:0/4075424841 reap_dead start 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf70c3700 1 -- 172.21.15.144:0/4075424841 shutdown_connections 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf70c3700 1 -- 172.21.15.144:0/4075424841 >> 172.21.15.144:0/4075424841 conn(0x7f7bf00d0000 msgr2=0x7f7bf00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf70c3700 1 -- 172.21.15.144:0/4075424841 shutdown_connections 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.667+0000 7f7bf70c3700 1 -- 172.21.15.144:0/4075424841 wait complete. 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.668+0000 7f7bf70c3700 1 Processor -- start 2023-11-28T11:56:21.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.668+0000 7f7bf70c3700 1 -- start start 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf70c3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0003f80 0x7f7bf00c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf70c3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf015ae80 0x7f7bf014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf70c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 0x7f7bf006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf70c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00c9020 con 0x7f7bf00c70d0 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf70c3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00c7a40 con 0x7f7be0003f80 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf70c3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00c7d20 con 0x7f7bf015ae80 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf5660700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 0x7f7bf006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf5660700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 0x7f7bf006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39176/0 (socket says 172.21.15.144:39176) 2023-11-28T11:56:21.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf5660700 1 -- 172.21.15.144:0/3766232172 learned_addr learned my addr 172.21.15.144:0/3766232172 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7bf4e5f700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0003f80 0x7f7bf00c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.669+0000 7f7beffff700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf015ae80 0x7f7bf014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf5660700 1 -- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf015ae80 msgr2=0x7f7bf014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf5660700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf015ae80 0x7f7bf014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf5660700 1 -- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0003f80 msgr2=0x7f7bf00c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf5660700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0003f80 0x7f7bf00c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf5660700 1 -- 172.21.15.144:0/3766232172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf015dff0 con 0x7f7bf00c70d0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf5660700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 0x7f7bf006a270 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f7bf009e140 tx=0x7f7be80104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be801b070 con 0x7f7bf00c70d0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be800f040 con 0x7f7bf00c70d0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf015dd10 con 0x7f7bf00c70d0 2023-11-28T11:56:21.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be800e050 con 0x7f7bf00c70d0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.670+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf00af400 con 0x7f7bf00c70d0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.671+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf00d0e60 con 0x7f7bf00c70d0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.671+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7be8010d00 con 0x7f7bf00c70d0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.672+0000 7f7bed7fa700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7bd40682f0 0x7f7bd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.672+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7be8085f30 con 0x7f7bf00c70d0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.672+0000 7f7bf4e5f700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7bd40682f0 0x7f7bd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.672+0000 7f7bf4e5f700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7bd40682f0 0x7f7bd406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4001c30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:21.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.676+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7be8052700 con 0x7f7bf00c70d0 2023-11-28T11:56:21.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.858+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bf006dca0 con 0x7f7bf00c70d0 2023-11-28T11:56:21.859 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.859+0000 7f7bed7fa700 1 -- 172.21.15.144:0/3766232172 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7be8055d20 con 0x7f7bf00c70d0 2023-11-28T11:56:21.859 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:56:21.861 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7bd40682f0 msgr2=0x7f7bd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.861 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7bd40682f0 0x7f7bd406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4001c30 comp rx=0 tx=0).stop 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 msgr2=0x7f7bf006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 0x7f7bf006a270 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f7bf009e140 tx=0x7f7be80104a0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 shutdown_connections 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf00c70d0 0x7f7bf006a270 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f7bd40682f0 0x7f7bd406a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0003f80 0x7f7bf00c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 --2- 172.21.15.144:0/3766232172 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7bf015ae80 0x7f7bf014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 >> 172.21.15.144:0/3766232172 conn(0x7f7bf00d0000 msgr2=0x7f7bf00b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 shutdown_connections 2023-11-28T11:56:21.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.861+0000 7f7bf70c3700 1 -- 172.21.15.144:0/3766232172 wait complete. 2023-11-28T11:56:21.873 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:56:21.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.995+0000 7f133d553700 1 Processor -- start 2023-11-28T11:56:21.996 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.995+0000 7f133d553700 1 -- start start 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f133d553700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1338071040 0x7f1338071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f133d553700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133806dee0 0x7f133806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f133d553700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1338071950 con 0x7f133806cdc0 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f133d553700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1338161550 con 0x7f1338071040 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f133d553700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133815d060 con 0x7f133806dee0 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f13377fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f133806cdc0 0x7f133806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55892/0 (socket says 172.21.15.144:55892) 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f13377fe700 1 -- 172.21.15.144:0/3841847559 learned_addr learned my addr 172.21.15.144:0/3841847559 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f13367fc700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133806dee0 0x7f133806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.996+0000 7f1336ffd700 1 --2- 172.21.15.144:0/3841847559 >> v2:172.21.15.177:3300/0 conn(0x7f1338071040 0x7f1338071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1335ffb700 1 -- 172.21.15.144:0/3841847559 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721294405 0 0) 0x7f1338071950 con 0x7f133806cdc0 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1335ffb700 1 -- 172.21.15.144:0/3841847559 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13240035b0 con 0x7f133806cdc0 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1336ffd700 1 -- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133806dee0 msgr2=0x7f133806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1336ffd700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133806dee0 0x7f133806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1336ffd700 1 -- 172.21.15.144:0/3841847559 >> v1:172.21.15.144:6789/0 conn(0x7f133806cdc0 legacy=0x7f133806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1336ffd700 1 -- 172.21.15.144:0/3841847559 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133815d340 con 0x7f1338071040 2023-11-28T11:56:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f13367fc700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133806dee0 0x7f133806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1336ffd700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1338071040 0x7f1338071410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1320009840 tx=0x7f1320009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef741bc3a1c8d8c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1335ffb700 1 -- 172.21.15.144:0/3841847559 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320011830 con 0x7f1338071040 2023-11-28T11:56:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1335ffb700 1 -- 172.21.15.144:0/3841847559 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320011990 con 0x7f1338071040 2023-11-28T11:56:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.997+0000 7f1335ffb700 1 -- 172.21.15.144:0/3841847559 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320011d20 con 0x7f1338071040 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 -- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1338071040 msgr2=0x7f1338071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1338071040 0x7f1338071410 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1320009840 tx=0x7f1320009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 -- 172.21.15.144:0/3841847559 shutdown_connections 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1338071040 0x7f1338071410 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 --2- 172.21.15.144:0/3841847559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f133806dee0 0x7f133806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 -- 172.21.15.144:0/3841847559 >> 172.21.15.144:0/3841847559 conn(0x7f13380d0000 msgr2=0x7f13380d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 -- 172.21.15.144:0/3841847559 shutdown_connections 2023-11-28T11:56:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.998+0000 7f133d553700 1 -- 172.21.15.144:0/3841847559 wait complete. 2023-11-28T11:56:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.999+0000 7f133d553700 1 Processor -- start 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:21.999+0000 7f133d553700 1 -- start start 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f133d553700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 0x7f13381f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f133d553700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133806dee0 0x7f13381556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f133d553700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1338071040 0x7f1338145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f133d553700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338146ee0 con 0x7f133806cdc0 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f133d553700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338146920 con 0x7f133806dee0 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f133d553700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1338146c00 con 0x7f1338071040 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 0x7f13381f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 0x7f13381f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39184/0 (socket says 172.21.15.144:39184) 2023-11-28T11:56:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 -- 172.21.15.144:0/1235088850 learned_addr learned my addr 172.21.15.144:0/1235088850 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f13367fc700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133806dee0 0x7f13381556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f13377fe700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1338071040 0x7f1338145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 -- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1338071040 msgr2=0x7f1338145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1338071040 0x7f1338145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 -- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133806dee0 msgr2=0x7f13381556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133806dee0 0x7f13381556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 -- 172.21.15.144:0/1235088850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280094a0 con 0x7f133806cdc0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.000+0000 7f1336ffd700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 0x7f13381f5450 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1320000c00 tx=0x7f1320004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.002+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13200036a0 con 0x7f133806cdc0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.002+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132001f070 con 0x7f133806cdc0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.002+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13200094a0 con 0x7f133806cdc0 2023-11-28T11:56:22.008 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.002+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320003a60 con 0x7f133806cdc0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.002+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133815d900 con 0x7f133806cdc0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.003+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133815d620 con 0x7f133806cdc0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.004+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f132000f050 con 0x7f133806cdc0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.004+0000 7f132f7fe700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13180682f0 0x7f131806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.004+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f1320021070 con 0x7f133806cdc0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.004+0000 7f13367fc700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13180682f0 0x7f131806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.004+0000 7f13367fc700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13180682f0 0x7f131806a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13380cde20 tx=0x7f1328009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.008+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1320053c00 con 0x7f133806cdc0 2023-11-28T11:56:22.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.189+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f133806e870 con 0x7f133806cdc0 2023-11-28T11:56:22.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.189+0000 7f132f7fe700 1 -- 172.21.15.144:0/1235088850 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1320057220 con 0x7f133806cdc0 2023-11-28T11:56:22.190 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:56:22.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13180682f0 msgr2=0x7f131806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13180682f0 0x7f131806a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13380cde20 tx=0x7f1328009210 comp rx=0 tx=0).stop 2023-11-28T11:56:22.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 msgr2=0x7f13381f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 0x7f13381f5450 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1320000c00 tx=0x7f1320004750 comp rx=0 tx=0).stop 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 shutdown_connections 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133806cdc0 0x7f13381f5450 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f13180682f0 0x7f131806a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1338071040 0x7f1338145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 --2- 172.21.15.144:0/1235088850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f133806dee0 0x7f13381556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 >> 172.21.15.144:0/1235088850 conn(0x7f13380d0000 msgr2=0x7f133806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 shutdown_connections 2023-11-28T11:56:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.191+0000 7f133d553700 1 -- 172.21.15.144:0/1235088850 wait complete. 2023-11-28T11:56:22.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:56:22.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 Processor -- start 2023-11-28T11:56:22.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 -- start start 2023-11-28T11:56:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efcdc06ba90 0x7efcdc0cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 0x7efcdc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0cf5e0 con 0x7efcdc15ca50 2023-11-28T11:56:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcdc0cd470 con 0x7efcdc06ba90 2023-11-28T11:56:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.326+0000 7efce19b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0cecb0 con 0x7efcdc0ccc40 2023-11-28T11:56:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 0x7efcdc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcda7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efcdc06ba90 0x7efcdc0cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 0x7efcdc0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:51278/0 (socket says 172.21.15.144:51278) 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 -- 172.21.15.144:0/2264457093 learned_addr learned my addr 172.21.15.144:0/2264457093 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 -- 172.21.15.144:0/2264457093 >> v2:172.21.15.177:3300/0 conn(0x7efcdc06ba90 msgr2=0x7efcdc0cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 --2- 172.21.15.144:0/2264457093 >> v2:172.21.15.177:3300/0 conn(0x7efcdc06ba90 0x7efcdc0cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 -- 172.21.15.144:0/2264457093 >> v1:172.21.15.177:6789/0 conn(0x7efcdc15ca50 legacy=0x7efcdc06b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 -- 172.21.15.144:0/2264457093 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0cef90 con 0x7efcdc0ccc40 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.327+0000 7efcdb7fe700 1 --2- 172.21.15.144:0/2264457093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 0x7efcdc0cfa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009e50 tx=0x7efcd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e84ff23e103dc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efcd9ffb700 1 -- 172.21.15.144:0/2264457093 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd400e070 con 0x7efcdc0ccc40 2023-11-28T11:56:22.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efcd9ffb700 1 -- 172.21.15.144:0/2264457093 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd400b910 con 0x7efcdc0ccc40 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efcd9ffb700 1 -- 172.21.15.144:0/2264457093 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd400bcc0 con 0x7efcdc0ccc40 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 -- 172.21.15.144:0/2264457093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 msgr2=0x7efcdc0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 --2- 172.21.15.144:0/2264457093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 0x7efcdc0cfa70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009e50 tx=0x7efcd400b330 comp rx=0 tx=0).stop 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 -- 172.21.15.144:0/2264457093 shutdown_connections 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 --2- 172.21.15.144:0/2264457093 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc0ccc40 0x7efcdc0cfa70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 --2- 172.21.15.144:0/2264457093 >> v2:172.21.15.177:3300/0 conn(0x7efcdc06ba90 0x7efcdc0cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 -- 172.21.15.144:0/2264457093 >> 172.21.15.144:0/2264457093 conn(0x7efcdc09a230 msgr2=0x7efcdc08afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 -- 172.21.15.144:0/2264457093 shutdown_connections 2023-11-28T11:56:22.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.328+0000 7efce19b3700 1 -- 172.21.15.144:0/2264457093 wait complete. 2023-11-28T11:56:22.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.329+0000 7efce19b3700 1 Processor -- start 2023-11-28T11:56:22.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 -- start start 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc06ba90 0x7efcdc1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcdc0ccc40 0x7efcdc0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 0x7efcdc0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b5f20 con 0x7efcdc15ca50 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b1dd0 con 0x7efcdc0ccc40 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efce19b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0b6d40 con 0x7efcdc06ba90 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efcdb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 0x7efcdc0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.330+0000 7efcdb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 0x7efcdc0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39202/0 (socket says 172.21.15.144:39202) 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 -- 172.21.15.144:0/40831155 learned_addr learned my addr 172.21.15.144:0/40831155 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcda7fc700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcdc0ccc40 0x7efcdc0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdaffd700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc06ba90 0x7efcdc1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 -- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc06ba90 msgr2=0x7efcdc1f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc06ba90 0x7efcdc1f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 -- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcdc0ccc40 msgr2=0x7efcdc0b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcdc0ccc40 0x7efcdc0b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 -- 172.21.15.144:0/40831155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4009b30 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcdb7fe700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 0x7efcdc0b1890 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0cedf0 tx=0x7efcd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4004800 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4004960 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc0cd0a0 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd401c680 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.331+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc0cf270 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.332+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc0cd230 con 0x7efcdc15ca50 2023-11-28T11:56:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.333+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efcd402c070 con 0x7efcdc15ca50 2023-11-28T11:56:22.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.333+0000 7efcc3fff700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efcc80682f0 0x7efcc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.333+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efcd4087770 con 0x7efcdc15ca50 2023-11-28T11:56:22.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.333+0000 7efcdaffd700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efcc80682f0 0x7efcc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.337 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.333+0000 7efcdaffd700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efcc80682f0 0x7efcc806a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efccc007fd0 tx=0x7efccc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.337+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efcdc0cd230 con 0x7efcdc15ca50 2023-11-28T11:56:22.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.524+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcdc0cd230 con 0x7efcdc15ca50 2023-11-28T11:56:22.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.524+0000 7efcc3fff700 1 -- 172.21.15.144:0/40831155 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcdc0cd230 con 0x7efcdc15ca50 2023-11-28T11:56:22.525 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efcc80682f0 msgr2=0x7efcc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efcc80682f0 0x7efcc806a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efccc007fd0 tx=0x7efccc009040 comp rx=0 tx=0).stop 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 msgr2=0x7efcdc0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 0x7efcdc0b1890 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0cedf0 tx=0x7efcd4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 shutdown_connections 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcdc15ca50 0x7efcdc0b1890 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efcc80682f0 0x7efcc806a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcdc06ba90 0x7efcdc1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 --2- 172.21.15.144:0/40831155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcdc0ccc40 0x7efcdc0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.526+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 >> 172.21.15.144:0/40831155 conn(0x7efcdc09a230 msgr2=0x7efcdc1e5370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:22.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.527+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 shutdown_connections 2023-11-28T11:56:22.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.527+0000 7efce19b3700 1 -- 172.21.15.144:0/40831155 wait complete. 2023-11-28T11:56:22.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:56:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.660+0000 7f81d5ef1700 1 Processor -- start 2023-11-28T11:56:22.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.660+0000 7f81d5ef1700 1 -- start start 2023-11-28T11:56:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81d5ef1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d0071040 0x7f81d0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81d5ef1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f81d0160540 0x7f81d01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81d5ef1700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d0071950 con 0x7f81d006dee0 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81d5ef1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d015d060 con 0x7f81d0160540 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81d5ef1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d015d340 con 0x7f81d0071040 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81ceffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f81d006dee0 0x7f81d006e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55920/0 (socket says 172.21.15.144:55920) 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81ceffd700 1 -- 172.21.15.144:0/3135908995 learned_addr learned my addr 172.21.15.144:0/3135908995 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81cf7fe700 1 --2- 172.21.15.144:0/3135908995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d0071040 0x7f81d0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81cffff700 1 --2- 172.21.15.144:0/3135908995 >> v2:172.21.15.177:3300/0 conn(0x7f81d0160540 0x7f81d01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81ce7fc700 1 -- 172.21.15.144:0/3135908995 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471714656 0 0) 0x7f81d0071950 con 0x7f81d006dee0 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81ce7fc700 1 -- 172.21.15.144:0/3135908995 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b80035b0 con 0x7f81d006dee0 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81cf7fe700 1 -- 172.21.15.144:0/3135908995 >> v2:172.21.15.177:3300/0 conn(0x7f81d0160540 msgr2=0x7f81d01609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81cf7fe700 1 --2- 172.21.15.144:0/3135908995 >> v2:172.21.15.177:3300/0 conn(0x7f81d0160540 0x7f81d01609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81cf7fe700 1 -- 172.21.15.144:0/3135908995 >> v1:172.21.15.144:6789/0 conn(0x7f81d006dee0 legacy=0x7f81d006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.661+0000 7f81cf7fe700 1 -- 172.21.15.144:0/3135908995 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d015d620 con 0x7f81d0071040 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81cf7fe700 1 --2- 172.21.15.144:0/3135908995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d0071040 0x7f81d0071410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009840 tx=0x7f81c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4827fa69d0b3af9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81ce7fc700 1 -- 172.21.15.144:0/3135908995 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4011830 con 0x7f81d0071040 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81ce7fc700 1 -- 172.21.15.144:0/3135908995 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c4011990 con 0x7f81d0071040 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81ce7fc700 1 -- 172.21.15.144:0/3135908995 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4011d00 con 0x7f81d0071040 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 -- 172.21.15.144:0/3135908995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d0071040 msgr2=0x7f81d0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/3135908995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d0071040 0x7f81d0071410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009840 tx=0x7f81c4009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 -- 172.21.15.144:0/3135908995 shutdown_connections 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/3135908995 >> v2:172.21.15.177:3300/0 conn(0x7f81d0160540 0x7f81d01609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/3135908995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d0071040 0x7f81d0071410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 -- 172.21.15.144:0/3135908995 >> 172.21.15.144:0/3135908995 conn(0x7f81d00d0000 msgr2=0x7f81d00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 -- 172.21.15.144:0/3135908995 shutdown_connections 2023-11-28T11:56:22.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.662+0000 7f81d5ef1700 1 -- 172.21.15.144:0/3135908995 wait complete. 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.663+0000 7f81d5ef1700 1 Processor -- start 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 -- start start 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d006dee0 0x7f81d0152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 0x7f81d006a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d0160540 0x7f81d006aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d01530e0 con 0x7f81d0071040 2023-11-28T11:56:22.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00c53f0 con 0x7f81d0160540 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.664+0000 7f81d5ef1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00c5830 con 0x7f81d006dee0 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 0x7f81d006a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 0x7f81d006a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39214/0 (socket says 172.21.15.144:39214) 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 -- 172.21.15.144:0/2537643572 learned_addr learned my addr 172.21.15.144:0/2537643572 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81cffff700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d0160540 0x7f81d006aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81cf7fe700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d006dee0 0x7f81d0152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 -- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d006dee0 msgr2=0x7f81d0152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d006dee0 0x7f81d0152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 -- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d0160540 msgr2=0x7f81d006aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d0160540 0x7f81d006aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 -- 172.21.15.144:0/2537643572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40094a0 con 0x7f81d0071040 2023-11-28T11:56:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81ceffd700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 0x7f81d006a4f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c420 tx=0x7f81c000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0005d70 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c000ce30 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d015ccd0 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00072d0 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.665+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d015d900 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.666+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d006a270 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.667+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81c0019030 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.667+0000 7f81b7fff700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f81bc0682f0 0x7f81bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.667+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f81c0084e50 con 0x7f81d0071040 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.667+0000 7f81cf7fe700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f81bc0682f0 0x7f81bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.668+0000 7f81cf7fe700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f81bc0682f0 0x7f81bc06a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000c00 tx=0x7f81c401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:22.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.671+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81c004b5e0 con 0x7f81d0071040 2023-11-28T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.870+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f81d006b480 con 0x7f81d0071040 2023-11-28T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.870+0000 7f81b7fff700 1 -- 172.21.15.144:0/2537643572 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f81c0017070 con 0x7f81d0071040 2023-11-28T11:56:22.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f81bc0682f0 msgr2=0x7f81bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f81bc0682f0 0x7f81bc06a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000c00 tx=0x7f81c401c040 comp rx=0 tx=0).stop 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 msgr2=0x7f81d006a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 0x7f81d006a4f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c420 tx=0x7f81c000c730 comp rx=0 tx=0).stop 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 shutdown_connections 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f81bc0682f0 0x7f81bc06a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d0160540 0x7f81d006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.874 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d006dee0 0x7f81d0152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 --2- 172.21.15.144:0/2537643572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d0071040 0x7f81d006a4f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:22.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 >> 172.21.15.144:0/2537643572 conn(0x7f81d00d0000 msgr2=0x7f81d00cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:22.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 shutdown_connections 2023-11-28T11:56:22.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:22.872+0000 7f81d5ef1700 1 -- 172.21.15.144:0/2537643572 wait complete. 2023-11-28T11:56:22.875 INFO:teuthology.orchestra.run.smithi144.stderr:listed 0 entries 2023-11-28T11:56:22.885 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:56:22.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:22.885 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:56:22.940 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:56:22.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:22.940 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:56:22.969 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:56:22.969 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:22.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:56:22.989 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:56:22.989 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:23.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:56:23.028 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:56:23.029 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:23.049 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:56:23.049 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:56:23.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.187+0000 7fcf7a6c6700 1 Processor -- start 2023-11-28T11:56:23.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.187+0000 7fcf7a6c6700 1 -- start start 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf7a6c6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcf74154e70 0x7fcf740cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf7a6c6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcf74071750 0x7fcf7415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf7a6c6700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf74155240 con 0x7fcf740cf360 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf7a6c6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7415b100 con 0x7fcf74154e70 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf7a6c6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7415b3e0 con 0x7fcf74071750 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcf74154e70 0x7fcf740cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcf74154e70 0x7fcf740cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39226/0 (socket says 172.21.15.144:39226) 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf737fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcf740cf360 0x7fcf74071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55934/0 (socket says 172.21.15.144:55934) 2023-11-28T11:56:23.191 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 -- 172.21.15.144:0/1852750468 learned_addr learned my addr 172.21.15.144:0/1852750468 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf78c63700 1 --2- 172.21.15.144:0/1852750468 >> v2:172.21.15.177:3300/0 conn(0x7fcf74071750 0x7fcf7415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 -- 172.21.15.144:0/1852750468 >> v2:172.21.15.177:3300/0 conn(0x7fcf74071750 msgr2=0x7fcf7415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 --2- 172.21.15.144:0/1852750468 >> v2:172.21.15.177:3300/0 conn(0x7fcf74071750 0x7fcf7415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 -- 172.21.15.144:0/1852750468 >> v1:172.21.15.144:6789/0 conn(0x7fcf740cf360 legacy=0x7fcf74071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf73fff700 1 -- 172.21.15.144:0/1852750468 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7415b6c0 con 0x7fcf74154e70 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.188+0000 7fcf78c63700 1 --2- 172.21.15.144:0/1852750468 >> v2:172.21.15.177:3300/0 conn(0x7fcf74071750 0x7fcf7415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf73fff700 1 --2- 172.21.15.144:0/1852750468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74154e70 0x7fcf740cdbd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcf740cb0a0 tx=0x7fcf68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23660d5e7457e06d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf72ffd700 1 -- 172.21.15.144:0/1852750468 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800b870 con 0x7fcf74154e70 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf72ffd700 1 -- 172.21.15.144:0/1852750468 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6800b9d0 con 0x7fcf74154e70 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf72ffd700 1 -- 172.21.15.144:0/1852750468 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800bd60 con 0x7fcf74154e70 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/1852750468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74154e70 msgr2=0x7fcf740cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/1852750468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74154e70 0x7fcf740cdbd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcf740cb0a0 tx=0x7fcf68009bd0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/1852750468 shutdown_connections 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/1852750468 >> v2:172.21.15.177:3300/0 conn(0x7fcf74071750 0x7fcf7415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/1852750468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74154e70 0x7fcf740cdbd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/1852750468 >> 172.21.15.144:0/1852750468 conn(0x7fcf740d0000 msgr2=0x7fcf740d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/1852750468 shutdown_connections 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.189+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/1852750468 wait complete. 2023-11-28T11:56:23.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.190+0000 7fcf7a6c6700 1 Processor -- start 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 -- start start 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 0x7fcf7414bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf740cf360 0x7fcf7415bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf74154e70 0x7fcf7415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7415d360 con 0x7fcf74071750 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740b5270 con 0x7fcf74154e70 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf7a6c6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7415c750 con 0x7fcf740cf360 2023-11-28T11:56:23.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.191+0000 7fcf73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 0x7fcf7414bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 0x7fcf7414bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39228/0 (socket says 172.21.15.144:39228) 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 -- 172.21.15.144:0/826629084 learned_addr learned my addr 172.21.15.144:0/826629084 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf78c63700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf74154e70 0x7fcf7415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf737fe700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf740cf360 0x7fcf7415bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 -- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf740cf360 msgr2=0x7fcf7415bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf740cf360 0x7fcf7415bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 -- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf74154e70 msgr2=0x7fcf7415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf74154e70 0x7fcf7415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 -- 172.21.15.144:0/826629084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68009550 con 0x7fcf74071750 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf737fe700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf740cf360 0x7fcf7415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf73fff700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 0x7fcf7414bfb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcf68014040 tx=0x7fcf68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.197 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68004520 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6801e070 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf740af520 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680038c0 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.192+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7415b9a0 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.193+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7415adb0 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.193+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf6800f050 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.194+0000 7fcf70ff9700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf600682f0 0x7fcf6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.194+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fcf68020070 con 0x7fcf74071750 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.194+0000 7fcf737fe700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf600682f0 0x7fcf6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.194+0000 7fcf737fe700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf600682f0 0x7fcf6006a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf741f6d00 tx=0x7fcf64007dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.198+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf68053ba0 con 0x7fcf74071750 2023-11-28T11:56:23.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:56:23.369 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:56:23.430 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.430+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf740ce460 con 0x7fcf74071750 2023-11-28T11:56:23.431 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.430+0000 7fcf70ff9700 1 -- 172.21.15.144:0/826629084 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fcf680571c0 con 0x7fcf74071750 2023-11-28T11:56:23.431 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:23.431 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:54:05.730553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:56:23.433 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf600682f0 msgr2=0x7fcf6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf600682f0 0x7fcf6006a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcf741f6d00 tx=0x7fcf64007dd0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 msgr2=0x7fcf7414bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 0x7fcf7414bfb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcf68014040 tx=0x7fcf68004750 comp rx=0 tx=0).stop 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 shutdown_connections 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf74071750 0x7fcf7414bfb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf600682f0 0x7fcf6006a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf74154e70 0x7fcf7415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 --2- 172.21.15.144:0/826629084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf740cf360 0x7fcf7415bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 >> 172.21.15.144:0/826629084 conn(0x7fcf740d0000 msgr2=0x7fcf7406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 shutdown_connections 2023-11-28T11:56:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.433+0000 7fcf7a6c6700 1 -- 172.21.15.144:0/826629084 wait complete. 2023-11-28T11:56:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2023-11-28T11:56:23.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:56:23.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.606+0000 7fbcbeac1700 1 Processor -- start 2023-11-28T11:56:23.607 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.607+0000 7fbcbeac1700 1 -- start start 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.607+0000 7fbcbeac1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcb8071040 0x7fbcb8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.607+0000 7fbcbeac1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806dee0 0x7fbcb806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.607+0000 7fbcbeac1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb8071950 con 0x7fbcb806cdc0 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.607+0000 7fbcbeac1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb8161550 con 0x7fbcb8071040 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.607+0000 7fbcbeac1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb815d060 con 0x7fbcb806dee0 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbc85d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcb8071040 0x7fbcb8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbd05e700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbcb806cdc0 0x7fbcb806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:56442/0 (socket says 172.21.15.144:56442) 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbd05e700 1 -- 172.21.15.144:0/3476984288 learned_addr learned my addr 172.21.15.144:0/3476984288 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcb7fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806dee0 0x7fbcb806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbc85d700 1 -- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806dee0 msgr2=0x7fbcb806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbc85d700 1 --2- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806dee0 0x7fbcb806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbc85d700 1 -- 172.21.15.144:0/3476984288 >> v1:172.21.15.177:6789/0 conn(0x7fbcb806cdc0 legacy=0x7fbcb806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcb77fe700 1 -- 172.21.15.144:0/3476984288 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490403679 0 0) 0x7fbcb8071950 con 0x7fbcb806cdc0 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbc85d700 1 -- 172.21.15.144:0/3476984288 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb815d340 con 0x7fbcb8071040 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.608+0000 7fbcbc85d700 1 --2- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb8071410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009840 tx=0x7fbcac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25fff710f7a6401 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcb77fe700 1 -- 172.21.15.144:0/3476984288 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac011830 con 0x7fbcb8071040 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcb77fe700 1 -- 172.21.15.144:0/3476984288 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac011990 con 0x7fbcb8071040 2023-11-28T11:56:23.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcb77fe700 1 -- 172.21.15.144:0/3476984288 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac011d20 con 0x7fbcb8071040 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 -- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 msgr2=0x7fbcb8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb8071410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009840 tx=0x7fbcac009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 -- 172.21.15.144:0/3476984288 shutdown_connections 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb8071410 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/3476984288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806dee0 0x7fbcb806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 -- 172.21.15.144:0/3476984288 >> 172.21.15.144:0/3476984288 conn(0x7fbcb80d0000 msgr2=0x7fbcb80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 -- 172.21.15.144:0/3476984288 shutdown_connections 2023-11-28T11:56:23.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.609+0000 7fbcbeac1700 1 -- 172.21.15.144:0/3476984288 wait complete. 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.610+0000 7fbcbeac1700 1 Processor -- start 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.610+0000 7fbcbeac1700 1 -- start start 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbeac1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806cdc0 0x7fbcb81f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbeac1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 0x7fbcb80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbeac1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb806a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbeac1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb81545f0 con 0x7fbcb806dee0 2023-11-28T11:56:23.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbeac1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8155e00 con 0x7fbcb8071040 2023-11-28T11:56:23.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbeac1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb81560e0 con 0x7fbcb806cdc0 2023-11-28T11:56:23.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcb7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 0x7fbcb80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcb7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 0x7fbcb80b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39244/0 (socket says 172.21.15.144:39244) 2023-11-28T11:56:23.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcb7fff700 1 -- 172.21.15.144:0/4009842404 learned_addr learned my addr 172.21.15.144:0/4009842404 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:23.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbc85d700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806cdc0 0x7fbcb81f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.611+0000 7fbcbd05e700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb806a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb7fff700 1 -- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806cdc0 msgr2=0x7fbcb81f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb7fff700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806cdc0 0x7fbcb81f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb7fff700 1 -- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 msgr2=0x7fbcb806a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb7fff700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb806a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb7fff700 1 -- 172.21.15.144:0/4009842404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0094a0 con 0x7fbcb806dee0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb7fff700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 0x7fbcb80b54f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbca400e870 tx=0x7fbca400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4009590 con 0x7fbcb806dee0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca4013680 con 0x7fbcb806dee0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40139f0 con 0x7fbcb806dee0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb815d900 con 0x7fbcb806dee0 2023-11-28T11:56:23.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.612+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb815d620 con 0x7fbcb806dee0 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.613+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb806e870 con 0x7fbcb806dee0 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.613+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbca4019030 con 0x7fbcb806dee0 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.614+0000 7fbcb57fa700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc9c0682f0 0x7fbc9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.614+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbca40859e0 con 0x7fbcb806dee0 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.614+0000 7fbcbc85d700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc9c0682f0 0x7fbc9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.614+0000 7fbcbc85d700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc9c0682f0 0x7fbc9c06a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009fd0 tx=0x7fbcac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.618+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbca40521b0 con 0x7fbcb806dee0 2023-11-28T11:56:23.805 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.804+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcb80720a0 con 0x7fbcb806dee0 2023-11-28T11:56:23.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.805+0000 7fbcb57fa700 1 -- 172.21.15.144:0/4009842404 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18151 (secure 0 0 0) 0x7fbca40557d0 con 0x7fbcb806dee0 2023-11-28T11:56:23.806 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:23.806 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":62,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:54:06.735804+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc9c0682f0 msgr2=0x7fbc9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc9c0682f0 0x7fbc9c06a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009fd0 tx=0x7fbcac008030 comp rx=0 tx=0).stop 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 msgr2=0x7fbcb80b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 0x7fbcb80b54f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbca400e870 tx=0x7fbca400eb80 comp rx=0 tx=0).stop 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 shutdown_connections 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbc9c0682f0 0x7fbc9c06a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcb8071040 0x7fbcb806a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcb806cdc0 0x7fbcb81f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 --2- 172.21.15.144:0/4009842404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb806dee0 0x7fbcb80b54f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 >> 172.21.15.144:0/4009842404 conn(0x7fbcb80d0000 msgr2=0x7fbcb806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:23.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 shutdown_connections 2023-11-28T11:56:23.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.808+0000 7fbcbeac1700 1 -- 172.21.15.144:0/4009842404 wait complete. 2023-11-28T11:56:23.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:51:10.205259+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:51:10.523478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:53:51.132040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:53:52.295120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:56:23.823 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:56:23.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.945+0000 7fd6c8a42700 1 Processor -- start 2023-11-28T11:56:23.946 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.945+0000 7fd6c8a42700 1 -- start start 2023-11-28T11:56:23.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c8a42700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd6c4071040 0x7fd6c4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c8a42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4160540 0x7fd6c41609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c8a42700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c4071950 con 0x7fd6c406dee0 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c8a42700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c415d060 con 0x7fd6c4071040 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c8a42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c415d340 con 0x7fd6c4160540 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c1d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd6c406dee0 0x7fd6c406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:55962/0 (socket says 172.21.15.144:55962) 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c1d9b700 1 -- 172.21.15.144:0/1134692375 learned_addr learned my addr 172.21.15.144:0/1134692375 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c2d9d700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4160540 0x7fd6c41609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.946+0000 7fd6c259c700 1 --2- 172.21.15.144:0/1134692375 >> v2:172.21.15.177:3300/0 conn(0x7fd6c4071040 0x7fd6c4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c159a700 1 -- 172.21.15.144:0/1134692375 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354682999 0 0) 0x7fd6c4071950 con 0x7fd6c406dee0 2023-11-28T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c159a700 1 -- 172.21.15.144:0/1134692375 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ac0035b0 con 0x7fd6c406dee0 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c259c700 1 -- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4160540 msgr2=0x7fd6c41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c259c700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4160540 0x7fd6c41609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c259c700 1 -- 172.21.15.144:0/1134692375 >> v1:172.21.15.144:6789/0 conn(0x7fd6c406dee0 legacy=0x7fd6c406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c159a700 1 -- 172.21.15.144:0/1134692375 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683074912 0 0) 0x7fd6ac0035b0 con 0x7fd6c406dee0 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c259c700 1 -- 172.21.15.144:0/1134692375 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c415d620 con 0x7fd6c4071040 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c2d9d700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4160540 0x7fd6c41609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c259c700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4071040 0x7fd6c4071410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009840 tx=0x7fd6b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29d6fc131b5dbc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c159a700 1 -- 172.21.15.144:0/1134692375 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4001670 con 0x7fd6c4071040 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c159a700 1 -- 172.21.15.144:0/1134692375 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b8004030 con 0x7fd6c4071040 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.947+0000 7fd6c159a700 1 -- 172.21.15.144:0/1134692375 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80043c0 con 0x7fd6c4071040 2023-11-28T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 -- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4071040 msgr2=0x7fd6c4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4071040 0x7fd6c4071410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009840 tx=0x7fd6b8009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 -- 172.21.15.144:0/1134692375 shutdown_connections 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4160540 0x7fd6c41609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/1134692375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4071040 0x7fd6c4071410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 -- 172.21.15.144:0/1134692375 >> 172.21.15.144:0/1134692375 conn(0x7fd6c40d0000 msgr2=0x7fd6c40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 -- 172.21.15.144:0/1134692375 shutdown_connections 2023-11-28T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.948+0000 7fd6c8a42700 1 -- 172.21.15.144:0/1134692375 wait complete. 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.949+0000 7fd6c8a42700 1 Processor -- start 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.949+0000 7fd6c8a42700 1 -- start start 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c8a42700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 0x7fd6c40b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c8a42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4071040 0x7fd6c40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c8a42700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4160540 0x7fd6c4156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c8a42700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c40c4eb0 con 0x7fd6c406dee0 2023-11-28T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c8a42700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c40c6940 con 0x7fd6c4160540 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c8a42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c4156c60 con 0x7fd6c4071040 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 0x7fd6c40b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 0x7fd6c40b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39254/0 (socket says 172.21.15.144:39254) 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 -- 172.21.15.144:0/2202562222 learned_addr learned my addr 172.21.15.144:0/2202562222 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c2d9d700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4160540 0x7fd6c4156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c1d9b700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4071040 0x7fd6c40b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 -- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4071040 msgr2=0x7fd6c40b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4071040 0x7fd6c40b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 -- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4160540 msgr2=0x7fd6c4156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4160540 0x7fd6c4156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 -- 172.21.15.144:0/2202562222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc009a70 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.950+0000 7fd6c259c700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 0x7fd6c40b5750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80090b0 tx=0x7fd6b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.951+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8003bd0 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.951+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b801f070 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.951+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8011630 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.951+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b80094a0 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.951+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c415ccd0 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.952+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c415d900 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.952+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6b802a020 con 0x7fd6c406dee0 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.952+0000 7fd6b2ffd700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6a40682f0 0x7fd6a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:23.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.952+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd6b80870a0 con 0x7fd6c406dee0 2023-11-28T11:56:23.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.952+0000 7fd6c1d9b700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6a40682f0 0x7fd6a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:23.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.953+0000 7fd6c1d9b700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6a40682f0 0x7fd6a406a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4001eb0 tx=0x7fd6b4001c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:23.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:23.957+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6b8053810 con 0x7fd6c406dee0 2023-11-28T11:56:24.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.139+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6c406a270 con 0x7fd6c406dee0 2023-11-28T11:56:24.140 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.140+0000 7fd6b2ffd700 1 -- 172.21.15.144:0/2202562222 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6b8056e30 con 0x7fd6c406dee0 2023-11-28T11:56:24.141 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6a40682f0 msgr2=0x7fd6a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6a40682f0 0x7fd6a406a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4001eb0 tx=0x7fd6b4001c40 comp rx=0 tx=0).stop 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 msgr2=0x7fd6c40b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 0x7fd6c40b5750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80090b0 tx=0x7fd6b8004750 comp rx=0 tx=0).stop 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 shutdown_connections 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c406dee0 0x7fd6c40b5750 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd6a40682f0 0x7fd6a406a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6c4160540 0x7fd6c4156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 --2- 172.21.15.144:0/2202562222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6c4071040 0x7fd6c40b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 >> 172.21.15.144:0/2202562222 conn(0x7fd6c40d0000 msgr2=0x7fd6c415e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:24.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 shutdown_connections 2023-11-28T11:56:24.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.142+0000 7fd6c8a42700 1 -- 172.21.15.144:0/2202562222 wait complete. 2023-11-28T11:56:24.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:56:24.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.276+0000 7f7963095700 1 Processor -- start 2023-11-28T11:56:24.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.276+0000 7f7963095700 1 -- start start 2023-11-28T11:56:24.279 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7963095700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 0x7f795c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7963095700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7963095700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c0cc5f0 con 0x7f795c06b8c0 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7963095700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795c06bd40 con 0x7f795c15cef0 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7963095700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c15d470 con 0x7f795c0cbdf0 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f795bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7960e31700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 0x7f795c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f795bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:51346/0 (socket says 172.21.15.144:51346) 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f795bfff700 1 -- 172.21.15.144:0/1264925328 learned_addr learned my addr 172.21.15.144:0/1264925328 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7960e31700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 0x7f795c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:41536/0 (socket says 172.21.15.144:41536) 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.277+0000 7f7961632700 1 --1- 172.21.15.144:0/1264925328 >> v1:172.21.15.177:6789/0 conn(0x7f795c06b8c0 0x7f795c071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:56468/0 (socket says 172.21.15.144:56468) 2023-11-28T11:56:24.280 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795bfff700 1 -- 172.21.15.144:0/1264925328 >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 msgr2=0x7f795c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795bfff700 1 --2- 172.21.15.144:0/1264925328 >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 0x7f795c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795bfff700 1 -- 172.21.15.144:0/1264925328 >> v1:172.21.15.177:6789/0 conn(0x7f795c06b8c0 legacy=0x7f795c071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795bfff700 1 -- 172.21.15.144:0/1264925328 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c15d750 con 0x7f795c0cbdf0 2023-11-28T11:56:24.281 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795b7fe700 1 -- 172.21.15.144:0/1264925328 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564250017 0 0) 0x7f795c0cc5f0 con 0x7f795c06b8c0 2023-11-28T11:56:24.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f7960e31700 1 --2- 172.21.15.144:0/1264925328 >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 0x7f795c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:24.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795bfff700 1 --2- 172.21.15.144:0/1264925328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06b380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79480098f0 tx=0x7f7948009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c89c90d81b7ce64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795b7fe700 1 -- 172.21.15.144:0/1264925328 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948011830 con 0x7f795c0cbdf0 2023-11-28T11:56:24.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795b7fe700 1 -- 172.21.15.144:0/1264925328 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7948011990 con 0x7f795c0cbdf0 2023-11-28T11:56:24.282 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.278+0000 7f795b7fe700 1 -- 172.21.15.144:0/1264925328 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948011d20 con 0x7f795c0cbdf0 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 -- 172.21.15.144:0/1264925328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 msgr2=0x7f795c06b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 --2- 172.21.15.144:0/1264925328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06b380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79480098f0 tx=0x7f7948009c00 comp rx=0 tx=0).stop 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 -- 172.21.15.144:0/1264925328 shutdown_connections 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 --2- 172.21.15.144:0/1264925328 >> v2:172.21.15.177:3300/0 conn(0x7f795c15cef0 0x7f795c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 --2- 172.21.15.144:0/1264925328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06b380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 -- 172.21.15.144:0/1264925328 >> 172.21.15.144:0/1264925328 conn(0x7f795c0d0000 msgr2=0x7f795c0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 -- 172.21.15.144:0/1264925328 shutdown_connections 2023-11-28T11:56:24.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.279+0000 7f7963095700 1 -- 172.21.15.144:0/1264925328 wait complete. 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.280+0000 7f7963095700 1 Processor -- start 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.280+0000 7f7963095700 1 -- start start 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f795c06b8c0 0x7f795c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 0x7f795c06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0b5630 con 0x7f795c15cef0 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c1f6ee0 con 0x7f795c06b8c0 2023-11-28T11:56:24.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c1f6120 con 0x7f795c0cbdf0 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 0x7f795c06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 0x7f795c06a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39272/0 (socket says 172.21.15.144:39272) 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 -- 172.21.15.144:0/987938825 learned_addr learned my addr 172.21.15.144:0/987938825 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f795bfff700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7960e31700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f795c06b8c0 0x7f795c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 -- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 msgr2=0x7f795c06a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 -- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f795c06b8c0 msgr2=0x7f795c0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f795c06b8c0 0x7f795c0b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 -- 172.21.15.144:0/987938825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500094a0 con 0x7f795c15cef0 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7961632700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 0x7f795c06a9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f795400cca0 tx=0x7f795400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.287 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795400ab70 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795400acd0 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7948009550 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.281+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954007310 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.282+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c15dd10 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.282+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c15da30 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.283+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f795401a030 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.283+0000 7f79597fa700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79400682f0 0x7f794006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.283+0000 7f7960e31700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79400682f0 0x7f794006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.283+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f79540859c0 con 0x7f795c15cef0 2023-11-28T11:56:24.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.284+0000 7f7960e31700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79400682f0 0x7f794006a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f795c0b4020 tx=0x7f7950009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.287+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7954052130 con 0x7f795c15cef0 2023-11-28T11:56:24.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.469+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f795c15dff0 con 0x7f795c15cef0 2023-11-28T11:56:24.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.470+0000 7f79597fa700 1 -- 172.21.15.144:0/987938825 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79540180a0 con 0x7f795c15cef0 2023-11-28T11:56:24.471 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79400682f0 msgr2=0x7f794006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79400682f0 0x7f794006a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f795c0b4020 tx=0x7f7950009210 comp rx=0 tx=0).stop 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 msgr2=0x7f795c06a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 0x7f795c06a9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f795400cca0 tx=0x7f795400a060 comp rx=0 tx=0).stop 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 shutdown_connections 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795c15cef0 0x7f795c06a9c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.473 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79400682f0 0x7f794006a7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f795c06b8c0 0x7f795c0b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 --2- 172.21.15.144:0/987938825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f795c0cbdf0 0x7f795c06a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 >> 172.21.15.144:0/987938825 conn(0x7f795c0d0000 msgr2=0x7f795c0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:24.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 shutdown_connections 2023-11-28T11:56:24.474 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.472+0000 7f7963095700 1 -- 172.21.15.144:0/987938825 wait complete. 2023-11-28T11:56:24.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:56:24.608 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.607+0000 7fcfb611e700 1 Processor -- start 2023-11-28T11:56:24.609 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.607+0000 7fcfb611e700 1 -- start start 2023-11-28T11:56:24.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfb611e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcfb00cd8c0 0x7fcfb0140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfb611e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcfb00cdc90 0x7fcfb00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfb611e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00b2e70 0x7fcfb006f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfb611e700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb013ff00 con 0x7fcfb00cd8c0 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfb611e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb01404c0 con 0x7fcfb00cdc90 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfb611e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb015aa80 con 0x7fcfb00b2e70 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfaf7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcfb00cd8c0 0x7fcfb0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfaf7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcfb00cd8c0 0x7fcfb0140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39276/0 (socket says 172.21.15.144:39276) 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.608+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/333982089 learned_addr learned my addr 172.21.15.144:0/333982089 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/333982089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00b2e70 msgr2=0x7fcfb006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaf7fe700 1 --2- 172.21.15.144:0/333982089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00b2e70 0x7fcfb006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaeffd700 1 --2- 172.21.15.144:0/333982089 >> v2:172.21.15.177:3300/0 conn(0x7fcfb00cdc90 0x7fcfb00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/333982089 >> v2:172.21.15.177:3300/0 conn(0x7fcfb00cdc90 msgr2=0x7fcfb00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaf7fe700 1 --2- 172.21.15.144:0/333982089 >> v2:172.21.15.177:3300/0 conn(0x7fcfb00cdc90 0x7fcfb00b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/333982089 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb00b3310 con 0x7fcfb00cd8c0 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfaf7fe700 1 --2- 172.21.15.144:0/333982089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0140950 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0071c40 tx=0x7fcfa4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623d8bc9e45431c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfae7fc700 1 -- 172.21.15.144:0/333982089 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4005e30 con 0x7fcfb00cd8c0 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfae7fc700 1 -- 172.21.15.144:0/333982089 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4004030 con 0x7fcfb00cd8c0 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfae7fc700 1 -- 172.21.15.144:0/333982089 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40043e0 con 0x7fcfb00cd8c0 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.609+0000 7fcfb611e700 1 -- 172.21.15.144:0/333982089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00cd8c0 msgr2=0x7fcfb0140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 --2- 172.21.15.144:0/333982089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0140950 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0071c40 tx=0x7fcfa4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 -- 172.21.15.144:0/333982089 shutdown_connections 2023-11-28T11:56:24.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 --2- 172.21.15.144:0/333982089 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00b2e70 0x7fcfb006f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 --2- 172.21.15.144:0/333982089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0140950 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 --2- 172.21.15.144:0/333982089 >> v2:172.21.15.177:3300/0 conn(0x7fcfb00cdc90 0x7fcfb00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 -- 172.21.15.144:0/333982089 >> 172.21.15.144:0/333982089 conn(0x7fcfb006aed0 msgr2=0x7fcfb006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 -- 172.21.15.144:0/333982089 shutdown_connections 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.610+0000 7fcfb611e700 1 -- 172.21.15.144:0/333982089 wait complete. 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.611+0000 7fcfb611e700 1 Processor -- start 2023-11-28T11:56:24.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.611+0000 7fcfb611e700 1 -- start start 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 0x7fcfb015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00cdc90 0x7fcfb00ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00b24d0 con 0x7fcfb00b2e70 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb006c420 con 0x7fcfb00cd8c0 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb0143510 con 0x7fcfb00cdc90 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 0x7fcfb015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 0x7fcfb015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39282/0 (socket says 172.21.15.144:39282) 2023-11-28T11:56:24.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/404132067 learned_addr learned my addr 172.21.15.144:0/404132067 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaeffd700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaffff700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00cdc90 0x7fcfb00ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00cdc90 msgr2=0x7fcfb00ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00cdc90 0x7fcfb00ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb00cd8c0 msgr2=0x7fcfb0154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 -- 172.21.15.144:0/404132067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4009500 con 0x7fcfb00b2e70 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaeffd700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfaf7fe700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 0x7fcfb015ac20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009130 tx=0x7fcfa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4011620 con 0x7fcfb00b2e70 2023-11-28T11:56:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa40117a0 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4011b50 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb0140150 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.612+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb00b35f0 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.613+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb00ce100 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.614+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfa402c070 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.614+0000 7fcf97fff700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf9c0682f0 0x7fcf9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.614+0000 7fcfaeffd700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf9c0682f0 0x7fcf9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.614+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fcfa4086f70 con 0x7fcfb00b2e70 2023-11-28T11:56:24.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.615+0000 7fcfaeffd700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf9c0682f0 0x7fcf9c06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00cb290 tx=0x7fcfa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.618+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfa40b85e0 con 0x7fcfb00b2e70 2023-11-28T11:56:24.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.799+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfb004e720 con 0x7fcfb00b2e70 2023-11-28T11:56:24.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.799+0000 7fcf97fff700 1 -- 172.21.15.144:0/404132067 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfa4053740 con 0x7fcfb00b2e70 2023-11-28T11:56:24.800 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 32 2023-11-28T11:56:24.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf9c0682f0 msgr2=0x7fcf9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf9c0682f0 0x7fcf9c06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00cb290 tx=0x7fcfa0008040 comp rx=0 tx=0).stop 2023-11-28T11:56:24.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 msgr2=0x7fcfb015ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 0x7fcfb015ac20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009130 tx=0x7fcfa4004750 comp rx=0 tx=0).stop 2023-11-28T11:56:24.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 shutdown_connections 2023-11-28T11:56:24.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfb00b2e70 0x7fcfb015ac20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.803 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcf9c0682f0 0x7fcf9c06a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfb00cdc90 0x7fcfb00ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 --2- 172.21.15.144:0/404132067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb00cd8c0 0x7fcfb0154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.801+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 >> 172.21.15.144:0/404132067 conn(0x7fcfb006aed0 msgr2=0x7fcfb00cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:24.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.802+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 shutdown_connections 2023-11-28T11:56:24.804 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.802+0000 7fcfb611e700 1 -- 172.21.15.144:0/404132067 wait complete. 2023-11-28T11:56:24.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:56:24.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.936+0000 7f031eeae700 1 Processor -- start 2023-11-28T11:56:24.937 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.936+0000 7f031eeae700 1 -- start start 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f031eeae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 0x7f03180af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f031eeae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f031806ba90 0x7f03180ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f031eeae700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03180cc5f0 con 0x7f03180cbdf0 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f031eeae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031815d470 con 0x7f031806ba90 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f031eeae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031815d750 con 0x7f031815cef0 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f0317fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f03180cbdf0 0x7f031806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56004/0 (socket says 172.21.15.144:56004) 2023-11-28T11:56:24.948 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.937+0000 7f0317fff700 1 -- 172.21.15.144:0/2051810648 learned_addr learned my addr 172.21.15.144:0/2051810648 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:24.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515273800 0 0) 0x7f03180cc5f0 con 0x7f03180cbdf0 2023-11-28T11:56:24.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 0x7f03180af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03080035b0 con 0x7f03180cbdf0 2023-11-28T11:56:24.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031d44b700 1 --2- 172.21.15.144:0/2051810648 >> v2:172.21.15.177:3300/0 conn(0x7f031806ba90 0x7f03180ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3681190094 0 0) 0x7f03080035b0 con 0x7f03180cbdf0 2023-11-28T11:56:24.949 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 msgr2=0x7f03180af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 0x7f03180af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 >> v2:172.21.15.177:3300/0 conn(0x7f031806ba90 msgr2=0x7f03180ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 --2- 172.21.15.144:0/2051810648 >> v2:172.21.15.177:3300/0 conn(0x7f031806ba90 0x7f03180ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031815da30 con 0x7f03180cbdf0 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f0304001670 con 0x7f03180cbdf0 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 >> v1:172.21.15.144:6789/0 conn(0x7f03180cbdf0 legacy=0x7f031806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f0308003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0308004430 0x7f03080048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 0x7f0308004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031815d470 con 0x7f0308003a00 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031815d750 con 0x7f0308004430 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030800ba10 con 0x7f031815cef0 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f0308003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f0317fff700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0308004430 0x7f03080048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031d44b700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 0x7f0308004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 -- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 msgr2=0x7f0308004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f031815cef0 0x7f0308004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.951 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 -- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0308004430 msgr2=0x7f03080048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0308004430 0x7f03080048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 -- 172.21.15.144:0/2051810648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080035b0 con 0x7f0308003a00 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.938+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f0308003e10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f030c009b60 tx=0x7f0308005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752a93e027fc1f2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00e6c0 con 0x7f0308003a00 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00e820 con 0x7f0308003a00 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f03177fe700 1 -- 172.21.15.144:0/2051810648 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00eb90 con 0x7f0308003a00 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031eeae700 1 -- 172.21.15.144:0/2051810648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 msgr2=0x7f0308003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031eeae700 1 --2- 172.21.15.144:0/2051810648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f0308003e10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f030c009b60 tx=0x7f0308005880 comp rx=0 tx=0).stop 2023-11-28T11:56:24.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031d44b700 1 -- 172.21.15.144:0/2051810648 reap_dead start 2023-11-28T11:56:24.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031eeae700 1 -- 172.21.15.144:0/2051810648 shutdown_connections 2023-11-28T11:56:24.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031eeae700 1 -- 172.21.15.144:0/2051810648 >> 172.21.15.144:0/2051810648 conn(0x7f03180d0000 msgr2=0x7f03180d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:24.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031eeae700 1 -- 172.21.15.144:0/2051810648 shutdown_connections 2023-11-28T11:56:24.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.939+0000 7f031eeae700 1 -- 172.21.15.144:0/2051810648 wait complete. 2023-11-28T11:56:24.953 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.940+0000 7f031eeae700 1 Processor -- start 2023-11-28T11:56:24.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.940+0000 7f031eeae700 1 -- start start 2023-11-28T11:56:24.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031eeae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f03180c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.954 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031eeae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03180c7a90 0x7f0318159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031eeae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03180caa20 0x7f03180cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031eeae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318159a80 con 0x7f0308003a00 2023-11-28T11:56:24.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031eeae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c8d80 con 0x7f03180caa20 2023-11-28T11:56:24.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031eeae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c7f00 con 0x7f03180c7a90 2023-11-28T11:56:24.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031cc4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f03180c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.955 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031cc4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f03180c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39302/0 (socket says 172.21.15.144:39302) 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f031cc4a700 1 -- 172.21.15.144:0/2596708642 learned_addr learned my addr 172.21.15.144:0/2596708642 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.941+0000 7f0317fff700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03180c7a90 0x7f0318159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031d44b700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03180caa20 0x7f03180cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031cc4a700 1 -- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03180c7a90 msgr2=0x7f0318159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03180c7a90 0x7f0318159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031cc4a700 1 -- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03180caa20 msgr2=0x7f03180cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:24.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03180caa20 0x7f03180cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031cc4a700 1 -- 172.21.15.144:0/2596708642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031815dff0 con 0x7f0308003a00 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031cc4a700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f03180c9210 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e4b0 tx=0x7f030c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00e6c0 con 0x7f0308003a00 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c0038b0 con 0x7f0308003a00 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031815dd10 con 0x7f0308003a00 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c003c60 con 0x7f0308003a00 2023-11-28T11:56:24.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.942+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180af400 con 0x7f0308003a00 2023-11-28T11:56:24.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.943+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03180d0e60 con 0x7f0308003a00 2023-11-28T11:56:24.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.944+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f030c019030 con 0x7f0308003a00 2023-11-28T11:56:24.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.944+0000 7f03157fa700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02fc0682e0 0x7f02fc06a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:24.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.944+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f030c087380 con 0x7f0308003a00 2023-11-28T11:56:24.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.944+0000 7f0317fff700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02fc0682e0 0x7f02fc06a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:24.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.944+0000 7f0317fff700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02fc0682e0 0x7f02fc06a790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0304000c00 tx=0x7f0304007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:24.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:24.949+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f030c053b50 con 0x7f0308003a00 2023-11-28T11:56:25.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.132+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f031806f4a0 con 0x7f0308003a00 2023-11-28T11:56:25.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.132+0000 7f03157fa700 1 -- 172.21.15.144:0/2596708642 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f030c057170 con 0x7f0308003a00 2023-11-28T11:56:25.133 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 64 2023-11-28T11:56:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02fc0682e0 msgr2=0x7f02fc06a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02fc0682e0 0x7f02fc06a790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0304000c00 tx=0x7f0304007980 comp rx=0 tx=0).stop 2023-11-28T11:56:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 msgr2=0x7f03180c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f03180c9210 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e4b0 tx=0x7f030c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 shutdown_connections 2023-11-28T11:56:25.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308003a00 0x7f03180c9210 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f02fc0682e0 0x7f02fc06a790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03180caa20 0x7f03180cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 --2- 172.21.15.144:0/2596708642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03180c7a90 0x7f0318159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 >> 172.21.15.144:0/2596708642 conn(0x7f03180d0000 msgr2=0x7f031806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 shutdown_connections 2023-11-28T11:56:25.136 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.134+0000 7f031eeae700 1 -- 172.21.15.144:0/2596708642 wait complete. 2023-11-28T11:56:25.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:56:25.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.266+0000 7f710f727700 1 Processor -- start 2023-11-28T11:56:25.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.266+0000 7f710f727700 1 -- start start 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.267+0000 7f710f727700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 0x7f71080c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.267+0000 7f710f727700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 0x7f71080c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.267+0000 7f710f727700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080ccc00 con 0x7f7108159490 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.267+0000 7f710f727700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71080ccec0 con 0x7f71080b0d80 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.267+0000 7f710f727700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080cd1a0 con 0x7f7108159860 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710dcc4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 0x7f71080c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 0x7f71080c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.269 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710dcc4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 0x7f71080c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:41588/0 (socket says 172.21.15.144:41588) 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 0x7f71080c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:51394/0 (socket says 172.21.15.144:51394) 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710dcc4700 1 -- 172.21.15.144:0/740673500 learned_addr learned my addr 172.21.15.144:0/740673500 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710d4c3700 1 --1- 172.21.15.144:0/740673500 >> v1:172.21.15.177:6789/0 conn(0x7f7108159490 0x7f71080b0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:56512/0 (socket says 172.21.15.144:56512) 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 -- 172.21.15.144:0/740673500 >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 msgr2=0x7f71080c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 --2- 172.21.15.144:0/740673500 >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 0x7f71080c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 -- 172.21.15.144:0/740673500 >> v1:172.21.15.177:6789/0 conn(0x7f7108159490 legacy=0x7f71080b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 -- 172.21.15.144:0/740673500 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080cd480 con 0x7f7108159860 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710dcc4700 1 --2- 172.21.15.144:0/740673500 >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 0x7f71080c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f710ccc2700 1 --2- 172.21.15.144:0/740673500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 0x7f71080c94c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7104009840 tx=0x7f7104009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826b9188be933c47 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f70fffff700 1 -- 172.21.15.144:0/740673500 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011830 con 0x7f7108159860 2023-11-28T11:56:25.270 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f70fffff700 1 -- 172.21.15.144:0/740673500 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104011990 con 0x7f7108159860 2023-11-28T11:56:25.271 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.268+0000 7f70fffff700 1 -- 172.21.15.144:0/740673500 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011d20 con 0x7f7108159860 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 -- 172.21.15.144:0/740673500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 msgr2=0x7f71080c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 --2- 172.21.15.144:0/740673500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 0x7f71080c94c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7104009840 tx=0x7f7104009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 -- 172.21.15.144:0/740673500 shutdown_connections 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 --2- 172.21.15.144:0/740673500 >> v2:172.21.15.177:3300/0 conn(0x7f71080b0d80 0x7f71080c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 --2- 172.21.15.144:0/740673500 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7108159860 0x7f71080c94c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 -- 172.21.15.144:0/740673500 >> 172.21.15.144:0/740673500 conn(0x7f71080cb670 msgr2=0x7f71080cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:25.273 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 -- 172.21.15.144:0/740673500 shutdown_connections 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.269+0000 7f710f727700 1 -- 172.21.15.144:0/740673500 wait complete. 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.270+0000 7f710f727700 1 Processor -- start 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.270+0000 7f710f727700 1 -- start start 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710f727700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71080b0d80 0x7f71080c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710f727700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 0x7f7108155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710f727700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7108153310 0x7f71080aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710f727700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080c7640 con 0x7f7108159490 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710f727700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108156510 con 0x7f7108153310 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710f727700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080c6d00 con 0x7f71080b0d80 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 0x7f7108155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 0x7f7108155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39316/0 (socket says 172.21.15.144:39316) 2023-11-28T11:56:25.274 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 -- 172.21.15.144:0/3883455848 learned_addr learned my addr 172.21.15.144:0/3883455848 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710dcc4700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7108153310 0x7f71080aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710d4c3700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71080b0d80 0x7f71080c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 -- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71080b0d80 msgr2=0x7f71080c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71080b0d80 0x7f71080c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 -- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7108153310 msgr2=0x7f71080aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7108153310 0x7f71080aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.271+0000 7f710ccc2700 1 -- 172.21.15.144:0/3883455848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009b30 con 0x7f7108159490 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.272+0000 7f710ccc2700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 0x7f7108155930 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7104009fd0 tx=0x7f7104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.272+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040036a0 con 0x7f7108159490 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.272+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71040094a0 con 0x7f7108159490 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.272+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710401e070 con 0x7f7108159490 2023-11-28T11:56:25.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.272+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104003a40 con 0x7f7108159490 2023-11-28T11:56:25.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.272+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71080c9f40 con 0x7f7108159490 2023-11-28T11:56:25.276 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.273+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71080cd760 con 0x7f7108159490 2023-11-28T11:56:25.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.273+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f710400f050 con 0x7f7108159490 2023-11-28T11:56:25.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.274+0000 7f70fdffb700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f70ec0682f0 0x7f70ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.274+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7104086c60 con 0x7f7108159490 2023-11-28T11:56:25.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.274+0000 7f710d4c3700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f70ec0682f0 0x7f70ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.274+0000 7f710d4c3700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f70ec0682f0 0x7f70ec06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f60 tx=0x7f70f8007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.278+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f71040533b0 con 0x7f7108159490 2023-11-28T11:56:25.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.510+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71080cdc80 con 0x7f7108159490 2023-11-28T11:56:25.511 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.510+0000 7f70fdffb700 1 -- 172.21.15.144:0/3883455848 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f7104015090 con 0x7f7108159490 2023-11-28T11:56:25.511 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:25.511 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:54:05.730553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:56:25.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f70ec0682f0 msgr2=0x7f70ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f70ec0682f0 0x7f70ec06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000f60 tx=0x7f70f8007ed0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 msgr2=0x7f7108155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 0x7f7108155930 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7104009fd0 tx=0x7f7104004750 comp rx=0 tx=0).stop 2023-11-28T11:56:25.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 shutdown_connections 2023-11-28T11:56:25.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f70ec0682f0 0x7f70ec06a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7108153310 0x7f71080aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71080b0d80 0x7f71080c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 --2- 172.21.15.144:0/3883455848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7108159490 0x7f7108155930 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 >> 172.21.15.144:0/3883455848 conn(0x7f71080cb670 msgr2=0x7f71080703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 shutdown_connections 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.512+0000 7f710f727700 1 -- 172.21.15.144:0/3883455848 wait complete. 2023-11-28T11:56:25.514 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2023-11-28T11:56:25.525 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:56:25.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.645+0000 7fcc29ebd700 1 Processor -- start 2023-11-28T11:56:25.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.645+0000 7fcc29ebd700 1 -- start start 2023-11-28T11:56:25.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc29ebd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160740 0x7fcc240cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc29ebd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc24160b10 0x7fcc240cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc29ebd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2406bbf0 con 0x7fcc2406b060 2023-11-28T11:56:25.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc29ebd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240ca720 con 0x7fcc24160b10 2023-11-28T11:56:25.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc29ebd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240cc030 con 0x7fcc24160740 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc24160b10 0x7fcc240cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc237fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160740 0x7fcc240cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc24160b10 0x7fcc240cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.144:41598/0 (socket says 172.21.15.144:41598) 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 -- 172.21.15.144:0/3278616662 learned_addr learned my addr 172.21.15.144:0/3278616662 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc23fff700 1 --1- 172.21.15.144:0/3278616662 >> v1:172.21.15.177:6789/0 conn(0x7fcc2406b060 0x7fcc2406b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:56530/0 (socket says 172.21.15.144:56530) 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 -- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160740 msgr2=0x7fcc240cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 --2- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160740 0x7fcc240cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 -- 172.21.15.144:0/3278616662 >> v1:172.21.15.177:6789/0 conn(0x7fcc2406b060 legacy=0x7fcc2406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 -- 172.21.15.144:0/3278616662 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240c9e10 con 0x7fcc24160b10 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc22ffd700 1 -- 172.21.15.144:0/3278616662 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619613118 0 0) 0x7fcc2406bbf0 con 0x7fcc2406b060 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.646+0000 7fcc1affd700 1 --2- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160b10 0x7fcc240cd390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098f0 tx=0x7fcc0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce17db10c80269ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc22ffd700 1 -- 172.21.15.144:0/3278616662 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c011830 con 0x7fcc24160b10 2023-11-28T11:56:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc22ffd700 1 -- 172.21.15.144:0/3278616662 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c011990 con 0x7fcc24160b10 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc22ffd700 1 -- 172.21.15.144:0/3278616662 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c011d20 con 0x7fcc24160b10 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 -- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160b10 msgr2=0x7fcc240cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160b10 0x7fcc240cd390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0098f0 tx=0x7fcc0c009c00 comp rx=0 tx=0).stop 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 -- 172.21.15.144:0/3278616662 shutdown_connections 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160740 0x7fcc240cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/3278616662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160b10 0x7fcc240cd390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 -- 172.21.15.144:0/3278616662 >> 172.21.15.144:0/3278616662 conn(0x7fcc240b12b0 msgr2=0x7fcc240b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 -- 172.21.15.144:0/3278616662 shutdown_connections 2023-11-28T11:56:25.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.647+0000 7fcc29ebd700 1 -- 172.21.15.144:0/3278616662 wait complete. 2023-11-28T11:56:25.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.648+0000 7fcc29ebd700 1 Processor -- start 2023-11-28T11:56:25.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 -- start start 2023-11-28T11:56:25.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 0x7fcc241f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160740 0x7fcc24157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160b10 0x7fcc241577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc241f6720 con 0x7fcc2406b060 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc24160280 con 0x7fcc24160740 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc29ebd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc241f6360 con 0x7fcc24160b10 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.649+0000 7fcc237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 0x7fcc241f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 0x7fcc241f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39330/0 (socket says 172.21.15.144:39330) 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 -- 172.21.15.144:0/2051441946 learned_addr learned my addr 172.21.15.144:0/2051441946 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc1affd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160740 0x7fcc24157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc23fff700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160b10 0x7fcc241577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 -- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160b10 msgr2=0x7fcc241577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160b10 0x7fcc241577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 -- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160740 msgr2=0x7fcc24157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160740 0x7fcc24157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 -- 172.21.15.144:0/2051441946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c009550 con 0x7fcc2406b060 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc23fff700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160b10 0x7fcc241577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:25.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc237fe700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 0x7fcc241f6b20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9b0 tx=0x7fcc1400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14009710 con 0x7fcc2406b060 2023-11-28T11:56:25.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1400c850 con 0x7fcc2406b060 2023-11-28T11:56:25.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc240ca0d0 con 0x7fcc2406b060 2023-11-28T11:56:25.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400cbc0 con 0x7fcc2406b060 2023-11-28T11:56:25.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.650+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2406efc0 con 0x7fcc2406b060 2023-11-28T11:56:25.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.651+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc240ca3b0 con 0x7fcc2406b060 2023-11-28T11:56:25.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.652+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcc14019030 con 0x7fcc2406b060 2023-11-28T11:56:25.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.652+0000 7fcc20ff9700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc040682f0 0x7fcc0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.652+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fcc14085c40 con 0x7fcc2406b060 2023-11-28T11:56:25.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.652+0000 7fcc1affd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc040682f0 0x7fcc0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.652+0000 7fcc1affd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc040682f0 0x7fcc0406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c014040 tx=0x7fcc0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.656+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcc14052410 con 0x7fcc2406b060 2023-11-28T11:56:25.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.838+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc241f6990 con 0x7fcc2406b060 2023-11-28T11:56:25.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.839+0000 7fcc20ff9700 1 -- 172.21.15.144:0/2051441946 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18151 (secure 0 0 0) 0x7fcc14017090 con 0x7fcc2406b060 2023-11-28T11:56:25.840 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:25.840 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":62,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:54:06.735804+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc040682f0 msgr2=0x7fcc0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc040682f0 0x7fcc0406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c014040 tx=0x7fcc0c008030 comp rx=0 tx=0).stop 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 msgr2=0x7fcc241f6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 0x7fcc241f6b20 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e9b0 tx=0x7fcc1400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 shutdown_connections 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc2406b060 0x7fcc241f6b20 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fcc040682f0 0x7fcc0406a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc24160b10 0x7fcc241577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 --2- 172.21.15.144:0/2051441946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc24160740 0x7fcc24157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 >> 172.21.15.144:0/2051441946 conn(0x7fcc240b12b0 msgr2=0x7fcc240b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:25.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 shutdown_connections 2023-11-28T11:56:25.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.841+0000 7fcc29ebd700 1 -- 172.21.15.144:0/2051441946 wait complete. 2023-11-28T11:56:25.854 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:56:25.855 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:56:25.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.973+0000 7f759e2e9700 1 Processor -- start 2023-11-28T11:56:25.974 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.973+0000 7f759e2e9700 1 -- start start 2023-11-28T11:56:25.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f759e2e9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 0x7f75980af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f759e2e9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f759806ba90 0x7f75980ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f759e2e9700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75980cc5f0 con 0x7f75980cbdf0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f759e2e9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759815d470 con 0x7f759806ba90 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f759e2e9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759815d750 con 0x7f759815cef0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f75977fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f75980cbdf0 0x7f759806b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56052/0 (socket says 172.21.15.144:56052) 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.974+0000 7f75977fe700 1 -- 172.21.15.144:0/4284116906 learned_addr learned my addr 172.21.15.144:0/4284116906 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 0x7f75980af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f759c886700 1 --2- 172.21.15.144:0/4284116906 >> v2:172.21.15.177:3300/0 conn(0x7f759806ba90 0x7f75980ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375884193 0 0) 0x7f75980cc5f0 con 0x7f75980cbdf0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75800035b0 con 0x7f75980cbdf0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127826802 0 0) 0x7f75800035b0 con 0x7f75980cbdf0 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 msgr2=0x7f75980af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 0x7f75980af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 >> v2:172.21.15.177:3300/0 conn(0x7f759806ba90 msgr2=0x7f75980ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 --2- 172.21.15.144:0/4284116906 >> v2:172.21.15.177:3300/0 conn(0x7f759806ba90 0x7f75980ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759815da30 con 0x7f75980cbdf0 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f758c001670 con 0x7f75980cbdf0 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 >> v1:172.21.15.144:6789/0 conn(0x7f75980cbdf0 legacy=0x7f759806b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f7580003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 0x7f7580004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7580004d90 0x7f7580005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759815d470 con 0x7f7580003a00 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759815d750 con 0x7f7580004d90 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758000ba10 con 0x7f759815cef0 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f7580003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f759c886700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7580004d90 0x7f7580005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f75977fe700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 0x7f7580004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 -- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 msgr2=0x7f7580004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f759815cef0 0x7f7580004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 -- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7580004d90 msgr2=0x7f7580005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7580004d90 0x7f7580005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.975+0000 7f7597fff700 1 -- 172.21.15.144:0/4284116906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800035b0 con 0x7f7580003a00 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f7597fff700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f7580003e10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7588009b60 tx=0x7f7580005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743e6af38ac7445e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800e660 con 0x7f7580003a00 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758800e7c0 con 0x7f7580003a00 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f7596ffd700 1 -- 172.21.15.144:0/4284116906 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800eb30 con 0x7f7580003a00 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759e2e9700 1 -- 172.21.15.144:0/4284116906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 msgr2=0x7f7580003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759e2e9700 1 --2- 172.21.15.144:0/4284116906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f7580003e10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7588009b60 tx=0x7f7580005880 comp rx=0 tx=0).stop 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759c886700 1 -- 172.21.15.144:0/4284116906 reap_dead start 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759e2e9700 1 -- 172.21.15.144:0/4284116906 shutdown_connections 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759e2e9700 1 -- 172.21.15.144:0/4284116906 >> 172.21.15.144:0/4284116906 conn(0x7f75980d0000 msgr2=0x7f75980d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759e2e9700 1 -- 172.21.15.144:0/4284116906 shutdown_connections 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.976+0000 7f759e2e9700 1 -- 172.21.15.144:0/4284116906 wait complete. 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.977+0000 7f759e2e9700 1 Processor -- start 2023-11-28T11:56:25.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 -- start start 2023-11-28T11:56:25.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f759815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7598099a40 0x7f7598099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75980b3e80 0x7f75980b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759815b690 con 0x7f7580003a00 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759809a3f0 con 0x7f7598099a40 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f759e2e9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759814a580 con 0x7f75980b3e80 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f7597fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f759815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f7597fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f759815b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39350/0 (socket says 172.21.15.144:39350) 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.978+0000 7f7597fff700 1 -- 172.21.15.144:0/3892320366 learned_addr learned my addr 172.21.15.144:0/3892320366 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f75977fe700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7598099a40 0x7f7598099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f759c886700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75980b3e80 0x7f75980b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7597fff700 1 -- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75980b3e80 msgr2=0x7f75980b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7597fff700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75980b3e80 0x7f75980b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7597fff700 1 -- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7598099a40 msgr2=0x7f7598099eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7597fff700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7598099a40 0x7f7598099eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7597fff700 1 -- 172.21.15.144:0/3892320366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759815dff0 con 0x7f7580003a00 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f75977fe700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7598099a40 0x7f7598099eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7597fff700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f759815b150 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f758800e4b0 tx=0x7f75880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800e660 con 0x7f7580003a00 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588003ab0 con 0x7f7580003a00 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759815dd10 con 0x7f7580003a00 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588003e60 con 0x7f7580003a00 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.979+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75980af400 con 0x7f7580003a00 2023-11-28T11:56:25.986 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.980+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7598067600 con 0x7f7580003a00 2023-11-28T11:56:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.981+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7588019030 con 0x7f7580003a00 2023-11-28T11:56:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.981+0000 7f7594ff9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75840682e0 0x7f758406a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.981+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7588087bf0 con 0x7f7580003a00 2023-11-28T11:56:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.981+0000 7f75977fe700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75840682e0 0x7f758406a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.982+0000 7f75977fe700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75840682e0 0x7f758406a790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:25.987 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:25.985+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7588054460 con 0x7f7580003a00 2023-11-28T11:56:26.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.209+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7598147ef0 con 0x7f7580003a00 2023-11-28T11:56:26.210 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.209+0000 7f7594ff9700 1 -- 172.21.15.144:0/3892320366 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7f7588017090 con 0x7f7580003a00 2023-11-28T11:56:26.210 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:26.211 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:54:05.730553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:56:26.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75840682e0 msgr2=0x7f758406a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.212 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75840682e0 0x7f758406a790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f758c000c00 tx=0x7f758c009040 comp rx=0 tx=0).stop 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 msgr2=0x7f759815b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f759815b150 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f758800e4b0 tx=0x7f75880047c0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 shutdown_connections 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7580003a00 0x7f759815b150 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75840682e0 0x7f758406a790 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75980b3e80 0x7f75980b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.211+0000 7f759e2e9700 1 --2- 172.21.15.144:0/3892320366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7598099a40 0x7f7598099eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.212+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 >> 172.21.15.144:0/3892320366 conn(0x7f75980d0000 msgr2=0x7f759806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.212+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 shutdown_connections 2023-11-28T11:56:26.213 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.212+0000 7f759e2e9700 1 -- 172.21.15.144:0/3892320366 wait complete. 2023-11-28T11:56:26.214 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2023-11-28T11:56:26.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:56:26.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.343+0000 7fe5c952a700 1 Processor -- start 2023-11-28T11:56:26.344 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.343+0000 7fe5c952a700 1 -- start start 2023-11-28T11:56:26.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c952a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c4140950 0x7fe5c40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c952a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5c40b2e70 0x7fe5c406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c952a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c413ff00 con 0x7fe5c40cd8c0 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c952a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c41404c0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c952a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4140dc0 con 0x7fe5c4140950 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c37fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5c40b2e70 0x7fe5c406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c37fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5c40b2e70 0x7fe5c406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39360/0 (socket says 172.21.15.144:39360) 2023-11-28T11:56:26.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c37fe700 1 -- 172.21.15.144:0/379990813 learned_addr learned my addr 172.21.15.144:0/379990813 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:26.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.344+0000 7fe5c27fc700 1 --2- 172.21.15.144:0/379990813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c4140950 0x7fe5c40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c37fe700 1 -- 172.21.15.144:0/379990813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c4140950 msgr2=0x7fe5c40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:26.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c37fe700 1 --2- 172.21.15.144:0/379990813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c4140950 0x7fe5c40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c37fe700 1 -- 172.21.15.144:0/379990813 >> v1:172.21.15.177:6789/0 conn(0x7fe5c40cd8c0 legacy=0x7fe5c40cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c37fe700 1 -- 172.21.15.144:0/379990813 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c415aa80 con 0x7fe5c40b2e70 2023-11-28T11:56:26.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c37fe700 1 --2- 172.21.15.144:0/379990813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c406f850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009e50 tx=0x7fe5bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f75a0f3898083e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c1ffb700 1 -- 172.21.15.144:0/379990813 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc00e070 con 0x7fe5c40b2e70 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c1ffb700 1 -- 172.21.15.144:0/379990813 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc00b910 con 0x7fe5c40b2e70 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c1ffb700 1 -- 172.21.15.144:0/379990813 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc00bcc0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 -- 172.21.15.144:0/379990813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 msgr2=0x7fe5c406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 --2- 172.21.15.144:0/379990813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c406f850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009e50 tx=0x7fe5bc00b330 comp rx=0 tx=0).stop 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 -- 172.21.15.144:0/379990813 shutdown_connections 2023-11-28T11:56:26.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 --2- 172.21.15.144:0/379990813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c406f850 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 --2- 172.21.15.144:0/379990813 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c4140950 0x7fe5c40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 -- 172.21.15.144:0/379990813 >> 172.21.15.144:0/379990813 conn(0x7fe5c406aed0 msgr2=0x7fe5c406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:26.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.345+0000 7fe5c952a700 1 -- 172.21.15.144:0/379990813 shutdown_connections 2023-11-28T11:56:26.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.346+0000 7fe5c952a700 1 -- 172.21.15.144:0/379990813 wait complete. 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.347+0000 7fe5c952a700 1 Processor -- start 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.347+0000 7fe5c952a700 1 -- start start 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c952a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c952a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c40cd8c0 0x7fe5c415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c952a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4140950 0x7fe5c415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c952a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c415d010 con 0x7fe5c40b2e70 2023-11-28T11:56:26.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c952a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41594b0 con 0x7fe5c4140950 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c952a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40c6440 con 0x7fe5c40cd8c0 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c40cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39362/0 (socket says 172.21.15.144:39362) 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c2ffd700 1 -- 172.21.15.144:0/331540753 learned_addr learned my addr 172.21.15.144:0/331540753 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c37fe700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4140950 0x7fe5c415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.348+0000 7fe5c27fc700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c40cd8c0 0x7fe5c415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c2ffd700 1 -- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c40cd8c0 msgr2=0x7fe5c415bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c2ffd700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c40cd8c0 0x7fe5c415bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c2ffd700 1 -- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4140950 msgr2=0x7fe5c415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c2ffd700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4140950 0x7fe5c415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.356 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c2ffd700 1 -- 172.21.15.144:0/331540753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009b30 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c2ffd700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c40cce40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c4d0 tx=0x7fe5b800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80052d0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8005430 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c40b35f0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80038f0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.349+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c40b3310 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.350+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4140150 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.351+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe5b8019030 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.351+0000 7fe5ab7fe700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5b00682f0 0x7fe5b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.351+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe5b8084ff0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.351+0000 7fe5c27fc700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5b00682f0 0x7fe5b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.357 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.351+0000 7fe5c27fc700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5b00682f0 0x7fe5b006a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40cea30 tx=0x7fe5b4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.355+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5b80516e0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.580+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5c40b3ce0 con 0x7fe5c40b2e70 2023-11-28T11:56:26.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.580+0000 7fe5ab7fe700 1 -- 172.21.15.144:0/331540753 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4655 (secure 0 0 0) 0x7fe5b8017090 con 0x7fe5c40b2e70 2023-11-28T11:56:26.581 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:26.581 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:53:53.337934+0000","modified":"2023-11-28T11:54:05.730553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6837/3077123372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3077123372},{"type":"v1","addr":"172.21.15.144:6837","nonce":3077123372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:56:26.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5b00682f0 msgr2=0x7fe5b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5b00682f0 0x7fe5b006a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40cea30 tx=0x7fe5b4009210 comp rx=0 tx=0).stop 2023-11-28T11:56:26.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 msgr2=0x7fe5c40cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c40cce40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c4d0 tx=0x7fe5b800c7e0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 shutdown_connections 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5c40b2e70 0x7fe5c40cce40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fe5b00682f0 0x7fe5b006a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4140950 0x7fe5c415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 --2- 172.21.15.144:0/331540753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5c40cd8c0 0x7fe5c415bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 >> 172.21.15.144:0/331540753 conn(0x7fe5c406aed0 msgr2=0x7fe5c406bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.582+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 shutdown_connections 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.583+0000 7fe5c952a700 1 -- 172.21.15.144:0/331540753 wait complete. 2023-11-28T11:56:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2023-11-28T11:56:26.595 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:56:26.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 Processor -- start 2023-11-28T11:56:26.715 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 -- start start 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c158ce0 0x7f755c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f755c06fb30 0x7f755c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f755c0ce2e0 0x7f755c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c06bf10 con 0x7f755c0ce2e0 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755c15d730 con 0x7f755c06fb30 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.715+0000 7f756197d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0ce6f0 con 0x7f755c158ce0 2023-11-28T11:56:26.718 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f755c0ce2e0 0x7f755c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f755c0ce2e0 0x7f755c06c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39374/0 (socket says 172.21.15.144:39374) 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 -- 172.21.15.144:0/2420299749 learned_addr learned my addr 172.21.15.144:0/2420299749 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755affd700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c158ce0 0x7f755c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755a7fc700 1 --2- 172.21.15.144:0/2420299749 >> v2:172.21.15.177:3300/0 conn(0x7f755c06fb30 0x7f755c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 -- 172.21.15.144:0/2420299749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c158ce0 msgr2=0x7f755c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c158ce0 0x7f755c15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 -- 172.21.15.144:0/2420299749 >> v2:172.21.15.177:3300/0 conn(0x7f755c06fb30 msgr2=0x7f755c0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 --2- 172.21.15.144:0/2420299749 >> v2:172.21.15.177:3300/0 conn(0x7f755c06fb30 0x7f755c0cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 -- 172.21.15.144:0/2420299749 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c1590b0 con 0x7f755c0ce2e0 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755a7fc700 1 --2- 172.21.15.144:0/2420299749 >> v2:172.21.15.177:3300/0 conn(0x7f755c06fb30 0x7f755c0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755affd700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c158ce0 0x7f755c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:26.719 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f755b7fe700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c0ce2e0 0x7f755c06c3a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f755c0672e0 tx=0x7f755400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7d605118b1ca2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f7559ffb700 1 -- 172.21.15.144:0/2420299749 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755400e070 con 0x7f755c0ce2e0 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f7559ffb700 1 -- 172.21.15.144:0/2420299749 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755400b9c0 con 0x7f755c0ce2e0 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.716+0000 7f7559ffb700 1 -- 172.21.15.144:0/2420299749 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755400bd70 con 0x7f755c0ce2e0 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 -- 172.21.15.144:0/2420299749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c0ce2e0 msgr2=0x7f755c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c0ce2e0 0x7f755c06c3a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f755c0672e0 tx=0x7f755400b330 comp rx=0 tx=0).stop 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 -- 172.21.15.144:0/2420299749 shutdown_connections 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c0ce2e0 0x7f755c06c3a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 --2- 172.21.15.144:0/2420299749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c158ce0 0x7f755c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 --2- 172.21.15.144:0/2420299749 >> v2:172.21.15.177:3300/0 conn(0x7f755c06fb30 0x7f755c0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 -- 172.21.15.144:0/2420299749 >> 172.21.15.144:0/2420299749 conn(0x7f755c0cb670 msgr2=0x7f755c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:26.720 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 -- 172.21.15.144:0/2420299749 shutdown_connections 2023-11-28T11:56:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.717+0000 7f756197d700 1 -- 172.21.15.144:0/2420299749 wait complete. 2023-11-28T11:56:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.718+0000 7f756197d700 1 Processor -- start 2023-11-28T11:56:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.718+0000 7f756197d700 1 -- start start 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f756197d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c06fb30 0x7f755c14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f756197d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c0ce2e0 0x7f755c0c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f756197d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 0x7f755c0c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f756197d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c6a90 con 0x7f755c158ce0 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f756197d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c6d70 con 0x7f755c0ce2e0 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f756197d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0c7050 con 0x7f755c06fb30 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.719+0000 7f755b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 0x7f755c0c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 0x7f755c0c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39380/0 (socket says 172.21.15.144:39380) 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 -- 172.21.15.144:0/2401039440 learned_addr learned my addr 172.21.15.144:0/2401039440 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755a7fc700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c0ce2e0 0x7f755c0c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755affd700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c06fb30 0x7f755c14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 -- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c06fb30 msgr2=0x7f755c14e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c06fb30 0x7f755c14e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 -- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c0ce2e0 msgr2=0x7f755c0c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c0ce2e0 0x7f755c0c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 -- 172.21.15.144:0/2401039440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7554009b30 con 0x7f755c158ce0 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f755b7fe700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 0x7f755c0c2f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f755c067360 tx=0x7f7554004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554004250 con 0x7f755c158ce0 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75540043d0 con 0x7f755c158ce0 2023-11-28T11:56:26.725 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554004780 con 0x7f755c158ce0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c0cb3b0 con 0x7f755c158ce0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.720+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c0cb0d0 con 0x7f755c158ce0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.721+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c06b8c0 con 0x7f755c158ce0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.721+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f755402c070 con 0x7f755c158ce0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.722+0000 7f7543fff700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75480682f0 0x7f754806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.722+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f75540874f0 con 0x7f755c158ce0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.722+0000 7f755affd700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75480682f0 0x7f754806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:26.726 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.722+0000 7f755affd700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75480682f0 0x7f754806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f755c06bd20 tx=0x7f754c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:26.727 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.726+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7554053cc0 con 0x7f755c158ce0 2023-11-28T11:56:26.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.906+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f755c067390 con 0x7f755c158ce0 2023-11-28T11:56:26.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.908+0000 7f7543fff700 1 -- 172.21.15.144:0/2401039440 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18151 (secure 0 0 0) 0x7f75540572e0 con 0x7f755c158ce0 2023-11-28T11:56:26.908 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:26.909 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":62,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:54:06.735804+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:53:51.132040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T11:53:52.295120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.1","mappings":[{"from":4,"to":3}]},{"pgid":"10.23","mappings":[{"from":4,"to":5}]},{"pgid":"10.36","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75480682f0 msgr2=0x7f754806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75480682f0 0x7f754806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f755c06bd20 tx=0x7f754c008040 comp rx=0 tx=0).stop 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 msgr2=0x7f755c0c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 0x7f755c0c2f00 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f755c067360 tx=0x7f7554004ad0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 shutdown_connections 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f755c158ce0 0x7f755c0c2f00 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f75480682f0 0x7f754806a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f755c06fb30 0x7f755c14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 --2- 172.21.15.144:0/2401039440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f755c0ce2e0 0x7f755c0c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:26.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 >> 172.21.15.144:0/2401039440 conn(0x7f755c0cb670 msgr2=0x7f755c0ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:26.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 shutdown_connections 2023-11-28T11:56:26.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:26.910+0000 7f756197d700 1 -- 172.21.15.144:0/2401039440 wait complete. 2023-11-28T11:56:26.926 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:56:27.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.047+0000 7fbf5733a700 1 Processor -- start 2023-11-28T11:56:27.048 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.047+0000 7fbf5733a700 1 -- start start 2023-11-28T11:56:27.050 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf5733a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf500b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf5733a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbf50161340 0x7fbf500d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf5733a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbf500d2320 0x7fbf500cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf5733a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500b5800 con 0x7fbf50161340 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf5733a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500d1950 con 0x7fbf500d2320 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf5733a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500b5520 con 0x7fbf5015c9b0 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf548d5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbf50161340 0x7fbf500d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf548d5700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbf50161340 0x7fbf500d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39384/0 (socket says 172.21.15.144:39384) 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.048+0000 7fbf548d5700 1 -- 172.21.15.144:0/820584978 learned_addr learned my addr 172.21.15.144:0/820584978 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf550d6700 1 --2- 172.21.15.144:0/820584978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf500b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf558d7700 1 --2- 172.21.15.144:0/820584978 >> v2:172.21.15.177:3300/0 conn(0x7fbf500d2320 0x7fbf500cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf548d5700 1 -- 172.21.15.144:0/820584978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 msgr2=0x7fbf500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf548d5700 1 --2- 172.21.15.144:0/820584978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf500b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf548d5700 1 -- 172.21.15.144:0/820584978 >> v2:172.21.15.177:3300/0 conn(0x7fbf500d2320 msgr2=0x7fbf500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf548d5700 1 --2- 172.21.15.144:0/820584978 >> v2:172.21.15.177:3300/0 conn(0x7fbf500d2320 0x7fbf500cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf548d5700 1 -- 172.21.15.144:0/820584978 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500d1300 con 0x7fbf50161340 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf548d5700 1 --2- 172.21.15.144:0/820584978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf500d1de0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbf5008c4d0 tx=0x7fbf4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dd7e413fa59f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf47fff700 1 -- 172.21.15.144:0/820584978 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00b870 con 0x7fbf50161340 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf47fff700 1 -- 172.21.15.144:0/820584978 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c00b9d0 con 0x7fbf50161340 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.049+0000 7fbf47fff700 1 -- 172.21.15.144:0/820584978 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c00bd60 con 0x7fbf50161340 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 -- 172.21.15.144:0/820584978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 msgr2=0x7fbf500d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 --2- 172.21.15.144:0/820584978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf500d1de0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbf5008c4d0 tx=0x7fbf4c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 -- 172.21.15.144:0/820584978 shutdown_connections 2023-11-28T11:56:27.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 --2- 172.21.15.144:0/820584978 >> v2:172.21.15.177:3300/0 conn(0x7fbf500d2320 0x7fbf500cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 --2- 172.21.15.144:0/820584978 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf500b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 --2- 172.21.15.144:0/820584978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf500d1de0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 -- 172.21.15.144:0/820584978 >> 172.21.15.144:0/820584978 conn(0x7fbf50072600 msgr2=0x7fbf500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:27.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 -- 172.21.15.144:0/820584978 shutdown_connections 2023-11-28T11:56:27.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.050+0000 7fbf5733a700 1 -- 172.21.15.144:0/820584978 wait complete. 2023-11-28T11:56:27.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.051+0000 7fbf5733a700 1 Processor -- start 2023-11-28T11:56:27.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.051+0000 7fbf5733a700 1 -- start start 2023-11-28T11:56:27.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.051+0000 7fbf5733a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2320 0x7fbf501efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf50151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf501e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50152530 con 0x7fbf50161340 2023-11-28T11:56:27.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501f2a00 con 0x7fbf500d2320 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501f2ce0 con 0x7fbf5015c9b0 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf501e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf501e71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39394/0 (socket says 172.21.15.144:39394) 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 -- 172.21.15.144:0/239701571 learned_addr learned my addr 172.21.15.144:0/239701571 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf548d5700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf50151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf550d6700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2320 0x7fbf501efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 -- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 msgr2=0x7fbf50151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf50151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 -- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2320 msgr2=0x7fbf501efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2320 0x7fbf501efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:27.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 -- 172.21.15.144:0/239701571 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c009550 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf550d6700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2320 0x7fbf501efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf558d7700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf501e71b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800cc30 tx=0x7fbf4800a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800ab70 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4800acd0 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48003c40 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf500b5130 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.052+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf500d15e0 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.053+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf500b52c0 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.054+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf4801f070 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.054+0000 7fbf45ffb700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbf340682f0 0x7fbf3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.054+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fbf48085e60 con 0x7fbf50161340 2023-11-28T11:56:27.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.054+0000 7fbf550d6700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbf340682f0 0x7fbf3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:27.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.054+0000 7fbf550d6700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbf340682f0 0x7fbf3406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006fd0 tx=0x7fbf40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:27.060 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.058+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf480525d0 con 0x7fbf50161340 2023-11-28T11:56:27.283 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:27.282+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbf500745c0 con 0x7fbf50161340 2023-11-28T11:56:28.020 INFO:tasks.ceph.mon.a.smithi144.stderr:2023-11-28T11:56:28.019+0000 7fdb2a043700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:56:28.028 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.028+0000 7fbf45ffb700 1 -- 172.21.15.144:0/239701571 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbf48055bf0 con 0x7fbf50161340 2023-11-28T11:56:28.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbf340682f0 msgr2=0x7fbf3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:28.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbf340682f0 0x7fbf3406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006fd0 tx=0x7fbf40008040 comp rx=0 tx=0).stop 2023-11-28T11:56:28.032 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 msgr2=0x7fbf501e71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf501e71b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800cc30 tx=0x7fbf4800a1f0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 shutdown_connections 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50161340 0x7fbf501e71b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fbf340682f0 0x7fbf3406a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf500d2320 0x7fbf501efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 --2- 172.21.15.144:0/239701571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbf5015c9b0 0x7fbf50151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.033 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 >> 172.21.15.144:0/239701571 conn(0x7fbf50072600 msgr2=0x7fbf5015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:28.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 shutdown_connections 2023-11-28T11:56:28.034 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.031+0000 7fbf5733a700 1 -- 172.21.15.144:0/239701571 wait complete. 2023-11-28T11:56:28.034 INFO:teuthology.orchestra.run.smithi144.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:56:28.035 INFO:tasks.ceph.mds.a.smithi144.stderr: -6925> 2023-11-28T11:54:01.709+0000 7ff9a8fdc700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6147=0-4194304@1} caps={6147=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6147 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55be786a1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:56:28.047 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:56:28.071 INFO:tasks.ceph.mds.a.smithi144.stdout:starting mds.a at 2023-11-28T11:56:28.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.177+0000 7f1182410700 1 Processor -- start 2023-11-28T11:56:28.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.177+0000 7f1182410700 1 -- start start 2023-11-28T11:56:28.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f1182410700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0d01b0 0x7f117c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:28.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f1182410700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f117c0b16d0 0x7f117c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:28.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f1182410700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0b0230 con 0x7f117c0b20b0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f1182410700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c15cec0 con 0x7f117c0b16d0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f1182410700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b4570 con 0x7f117c0d01b0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117b7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f117c0b20b0 0x7f117c0b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56122/0 (socket says 172.21.15.144:56122) 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117b7fe700 1 -- 172.21.15.144:0/1941541330 learned_addr learned my addr 172.21.15.144:0/1941541330 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f11809ad700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f117c0b16d0 0x7f117c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117affd700 1 -- 172.21.15.144:0/1941541330 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998101697 0 0) 0x7f117c0b0230 con 0x7f117c0b20b0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117affd700 1 -- 172.21.15.144:0/1941541330 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11640035b0 con 0x7f117c0b20b0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117bfff700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0d01b0 0x7f117c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f11809ad700 1 -- 172.21.15.144:0/1941541330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0d01b0 msgr2=0x7f117c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f11809ad700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0d01b0 0x7f117c0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f11809ad700 1 -- 172.21.15.144:0/1941541330 >> v1:172.21.15.144:6789/0 conn(0x7f117c0b20b0 legacy=0x7f117c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f11809ad700 1 -- 172.21.15.144:0/1941541330 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0b4850 con 0x7f117c0b16d0 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117bfff700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0d01b0 0x7f117c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f11809ad700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0d1160 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1174009dc0 tx=0x7f117400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caf5fd008b98133 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117affd700 1 -- 172.21.15.144:0/1941541330 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400e070 con 0x7f117c0b16d0 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117affd700 1 -- 172.21.15.144:0/1941541330 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400bae0 con 0x7f117c0b16d0 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.178+0000 7f117affd700 1 -- 172.21.15.144:0/1941541330 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400be50 con 0x7f117c0b16d0 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 -- 172.21.15.144:0/1941541330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 msgr2=0x7f117c0d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0d1160 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1174009dc0 tx=0x7f117400b330 comp rx=0 tx=0).stop 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 -- 172.21.15.144:0/1941541330 shutdown_connections 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0d1160 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 --2- 172.21.15.144:0/1941541330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0d01b0 0x7f117c0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 -- 172.21.15.144:0/1941541330 >> 172.21.15.144:0/1941541330 conn(0x7f117c09c3f0 msgr2=0x7f117c0d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:28.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 -- 172.21.15.144:0/1941541330 shutdown_connections 2023-11-28T11:56:28.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.179+0000 7f1182410700 1 -- 172.21.15.144:0/1941541330 wait complete. 2023-11-28T11:56:28.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.180+0000 7f1182410700 1 Processor -- start 2023-11-28T11:56:28.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.180+0000 7f1182410700 1 -- start start 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f1182410700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f1182410700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0b20b0 0x7f117c0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f1182410700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c0d01b0 0x7f117c144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f1182410700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b6ed0 con 0x7f117c0b16d0 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f1182410700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b7ee0 con 0x7f117c0d01b0 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f1182410700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b7120 con 0x7f117c0b20b0 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f117bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f117bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39416/0 (socket says 172.21.15.144:39416) 2023-11-28T11:56:28.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f117bfff700 1 -- 172.21.15.144:0/162754217 learned_addr learned my addr 172.21.15.144:0/162754217 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f117b7fe700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0b20b0 0x7f117c0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.181+0000 7f11809ad700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c0d01b0 0x7f117c144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f117bfff700 1 -- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0b20b0 msgr2=0x7f117c0ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f117bfff700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0b20b0 0x7f117c0ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f117bfff700 1 -- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c0d01b0 msgr2=0x7f117c144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f117bfff700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c0d01b0 0x7f117c144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:28.187 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f117bfff700 1 -- 172.21.15.144:0/162754217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174009a70 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f117bfff700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0ce7e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c7f0 tx=0x7f116c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011800 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c011960 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c0afec0 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011cf0 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.182+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c0afbe0 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.183+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c0b2600 con 0x7f117c0b16d0 2023-11-28T11:56:28.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.183+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f116c021070 con 0x7f117c0b16d0 2023-11-28T11:56:28.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.183+0000 7f1178ff9700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f11680682f0 0x7f116806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:28.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.184+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f116c085b00 con 0x7f117c0b16d0 2023-11-28T11:56:28.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.184+0000 7f117b7fe700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f11680682f0 0x7f116806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:28.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.184+0000 7f117b7fe700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f11680682f0 0x7f116806a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f117c0b3430 tx=0x7f1170009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:28.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.188+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f116c052270 con 0x7f117c0b16d0 2023-11-28T11:56:28.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:28.413+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f117c0b7d30 con 0x7f117c0b16d0 2023-11-28T11:56:29.038 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.037+0000 7f1178ff9700 1 -- 172.21.15.144:0/162754217 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f116c055890 con 0x7f117c0b16d0 2023-11-28T11:56:29.040 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f11680682f0 msgr2=0x7f116806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f11680682f0 0x7f116806a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f117c0b3430 tx=0x7f1170009040 comp rx=0 tx=0).stop 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 msgr2=0x7f117c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0ce7e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c7f0 tx=0x7f116c00cb00 comp rx=0 tx=0).stop 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 shutdown_connections 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117c0b16d0 0x7f117c0ce7e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f11680682f0 0x7f116806a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f117c0d01b0 0x7f117c144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 --2- 172.21.15.144:0/162754217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f117c0b20b0 0x7f117c0ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 >> 172.21.15.144:0/162754217 conn(0x7f117c09c3f0 msgr2=0x7f117c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:29.041 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 shutdown_connections 2023-11-28T11:56:29.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.040+0000 7f1182410700 1 -- 172.21.15.144:0/162754217 wait complete. 2023-11-28T11:56:29.056 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:56:29.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.227+0000 7f2d53765700 1 Processor -- start 2023-11-28T11:56:29.228 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.228+0000 7f2d53765700 1 -- start start 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.228+0000 7f2d53765700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d53765700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d4c0af430 0x7f2d4c06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d53765700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0713c0 con 0x7f2d4c06a270 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d53765700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c06dee0 con 0x7f2d4c0af430 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d53765700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0716a0 con 0x7f2d4c0af060 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d4c0af430 0x7f2d4c06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:29.231 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d51d02700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f2d4c06a270 0x7f2d4c160590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56140/0 (socket says 172.21.15.144:56140) 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d51d02700 1 -- 172.21.15.144:0/5960226 learned_addr learned my addr 172.21.15.144:0/5960226 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d4c0af430 0x7f2d4c06d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39426/0 (socket says 172.21.15.144:39426) 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d51501700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 -- 172.21.15.144:0/5960226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 msgr2=0x7f2d4c070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 -- 172.21.15.144:0/5960226 >> v1:172.21.15.144:6789/0 conn(0x7f2d4c06a270 legacy=0x7f2d4c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d43fff700 1 -- 172.21.15.144:0/5960226 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224779930 0 0) 0x7f2d4c0713c0 con 0x7f2d4c06a270 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 -- 172.21.15.144:0/5960226 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c071980 con 0x7f2d4c0af430 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d51501700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d50d00700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c0af430 0x7f2d4c06d9a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009840 tx=0x7f2d48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f56530f9b7648f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d43fff700 1 -- 172.21.15.144:0/5960226 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48011830 con 0x7f2d4c0af430 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d43fff700 1 -- 172.21.15.144:0/5960226 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d48011990 con 0x7f2d4c0af430 2023-11-28T11:56:29.232 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.229+0000 7f2d43fff700 1 -- 172.21.15.144:0/5960226 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48011d20 con 0x7f2d4c0af430 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 -- 172.21.15.144:0/5960226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c0af430 msgr2=0x7f2d4c06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c0af430 0x7f2d4c06d9a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009840 tx=0x7f2d48009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 -- 172.21.15.144:0/5960226 shutdown_connections 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 --2- 172.21.15.144:0/5960226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c0af430 0x7f2d4c06d9a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 -- 172.21.15.144:0/5960226 >> 172.21.15.144:0/5960226 conn(0x7f2d4c06b380 msgr2=0x7f2d4c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 -- 172.21.15.144:0/5960226 shutdown_connections 2023-11-28T11:56:29.233 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.230+0000 7f2d53765700 1 -- 172.21.15.144:0/5960226 wait complete. 2023-11-28T11:56:29.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.231+0000 7f2d53765700 1 Processor -- start 2023-11-28T11:56:29.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.231+0000 7f2d53765700 1 -- start start 2023-11-28T11:56:29.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d53765700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 0x7f2d4c0c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:29.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d53765700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:29.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d53765700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d4c0af430 0x7f2d4c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:29.236 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d53765700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c4680 con 0x7f2d4c06a270 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d53765700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c3a70 con 0x7f2d4c0af430 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d53765700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0c3d50 con 0x7f2d4c0af060 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 0x7f2d4c0c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 0x7f2d4c0c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39430/0 (socket says 172.21.15.144:39430) 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 -- 172.21.15.144:0/3622026058 learned_addr learned my addr 172.21.15.144:0/3622026058 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d50d00700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51d02700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d4c0af430 0x7f2d4c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 -- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 msgr2=0x7f2d4c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 -- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d4c0af430 msgr2=0x7f2d4c15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d4c0af430 0x7f2d4c15cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 -- 172.21.15.144:0/3622026058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480094a0 con 0x7f2d4c06a270 2023-11-28T11:56:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d50d00700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51d02700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d4c0af430 0x7f2d4c15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.232+0000 7f2d51501700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 0x7f2d4c0c53d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c750 tx=0x7f2d3c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.233+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c011800 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.233+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c011960 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.233+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c071f40 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.233+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c011cd0 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.233+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c071c60 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.234+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c0cb670 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.234+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d3c020070 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.234+0000 7f2d41ffb700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d30068320 0x7f2d3006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.234+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2d3c085bc0 con 0x7f2d4c06a270 2023-11-28T11:56:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.234+0000 7f2d50d00700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d30068320 0x7f2d3006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:29.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.235+0000 7f2d50d00700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d30068320 0x7f2d3006a7d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0af820 tx=0x7f2d48008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:29.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.239+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d3c052330 con 0x7f2d4c06a270 2023-11-28T11:56:29.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:29.421+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d4c0745c0 con 0x7f2d4c06a270 2023-11-28T11:56:30.094 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.094+0000 7f2d41ffb700 1 -- 172.21.15.144:0/3622026058 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2d3c01e0d0 con 0x7f2d4c06a270 2023-11-28T11:56:30.097 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d30068320 msgr2=0x7f2d3006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:30.097 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d30068320 0x7f2d3006a7d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0af820 tx=0x7f2d48008030 comp rx=0 tx=0).stop 2023-11-28T11:56:30.097 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 msgr2=0x7f2d4c0c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:30.097 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 0x7f2d4c0c53d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00c750 tx=0x7f2d3c00cb10 comp rx=0 tx=0).stop 2023-11-28T11:56:30.097 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 shutdown_connections 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d4c06a270 0x7f2d4c0c53d0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f2d30068320 0x7f2d3006a7d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2d4c0af430 0x7f2d4c15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 --2- 172.21.15.144:0/3622026058 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2d4c0af060 0x7f2d4c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.096+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 >> 172.21.15.144:0/3622026058 conn(0x7f2d4c06b380 msgr2=0x7f2d4c0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.097+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 shutdown_connections 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.097+0000 7f2d53765700 1 -- 172.21.15.144:0/3622026058 wait complete. 2023-11-28T11:56:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:56:30.111 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:56:30.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.238+0000 7fa2fea1f700 1 Processor -- start 2023-11-28T11:56:30.239 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.239+0000 7fa2fea1f700 1 -- start start 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.239+0000 7fa2fea1f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa2f815a760 0x7fa2f8161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.239+0000 7fa2fea1f700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa2f80cded0 0x7fa2f80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.239+0000 7fa2fea1f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f815e0a0 con 0x7fa2f815dc30 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.239+0000 7fa2fea1f700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f815ab30 con 0x7fa2f80cded0 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.239+0000 7fa2fea1f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f8160d70 con 0x7fa2f815a760 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa2f80cded0 0x7fa2f80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa2f80cded0 0x7fa2f80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39442/0 (socket says 172.21.15.144:39442) 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/556645557 learned_addr learned my addr 172.21.15.144:0/556645557 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2f7fff700 1 --2- 172.21.15.144:0/556645557 >> v2:172.21.15.177:3300/0 conn(0x7fa2f815a760 0x7fa2f8161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/556645557 >> v2:172.21.15.177:3300/0 conn(0x7fa2f815a760 msgr2=0x7fa2f8161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:30.242 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 --2- 172.21.15.144:0/556645557 >> v2:172.21.15.177:3300/0 conn(0x7fa2f815a760 0x7fa2f8161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/556645557 >> v1:172.21.15.177:6789/0 conn(0x7fa2f815dc30 legacy=0x7fa2f80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/556645557 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80b5130 con 0x7fa2f80cded0 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2f7fff700 1 --2- 172.21.15.144:0/556645557 >> v2:172.21.15.177:3300/0 conn(0x7fa2f815a760 0x7fa2f8161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2fcfbc700 1 --2- 172.21.15.144:0/556645557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f80cded0 0x7fa2f80b4bf0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa2f808bff0 tx=0x7fa2f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf099cc54403c673 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2f6ffd700 1 -- 172.21.15.144:0/556645557 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000e070 con 0x7fa2f80cded0 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2f6ffd700 1 -- 172.21.15.144:0/556645557 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0003a20 con 0x7fa2f80cded0 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.240+0000 7fa2f6ffd700 1 -- 172.21.15.144:0/556645557 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0003dd0 con 0x7fa2f80cded0 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 -- 172.21.15.144:0/556645557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f80cded0 msgr2=0x7fa2f80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/556645557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f80cded0 0x7fa2f80b4bf0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa2f808bff0 tx=0x7fa2f000b330 comp rx=0 tx=0).stop 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 -- 172.21.15.144:0/556645557 shutdown_connections 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/556645557 >> v2:172.21.15.177:3300/0 conn(0x7fa2f815a760 0x7fa2f8161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/556645557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f80cded0 0x7fa2f80b4bf0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.243 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 -- 172.21.15.144:0/556645557 >> 172.21.15.144:0/556645557 conn(0x7fa2f8072620 msgr2=0x7fa2f80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:30.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 -- 172.21.15.144:0/556645557 shutdown_connections 2023-11-28T11:56:30.244 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.241+0000 7fa2fea1f700 1 -- 172.21.15.144:0/556645557 wait complete. 2023-11-28T11:56:30.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.242+0000 7fa2fea1f700 1 Processor -- start 2023-11-28T11:56:30.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.242+0000 7fa2fea1f700 1 -- start start 2023-11-28T11:56:30.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fea1f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f80cded0 0x7fa2f80cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:30.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fea1f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f815a760 0x7fa2f80ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:30.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fea1f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 0x7fa2f815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:30.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fea1f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80caa90 con 0x7fa2f815dc30 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fea1f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80cad70 con 0x7fa2f815a760 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fea1f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80cb050 con 0x7fa2f80cded0 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fcfbc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 0x7fa2f815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fcfbc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 0x7fa2f815f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39448/0 (socket says 172.21.15.144:39448) 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.243+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/3339581249 learned_addr learned my addr 172.21.15.144:0/3339581249 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2f7fff700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f80cded0 0x7fa2f80cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2f77fe700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f815a760 0x7fa2f80ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f80cded0 msgr2=0x7fa2f80cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fcfbc700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f80cded0 0x7fa2f80cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f815a760 msgr2=0x7fa2f80ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fcfbc700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f815a760 0x7fa2f80ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fcfbc700 1 -- 172.21.15.144:0/3339581249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f0009b30 con 0x7fa2f815dc30 2023-11-28T11:56:30.247 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fcfbc700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 0x7fa2f815f7b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0015040 tx=0x7fa2f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:30.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0004440 con 0x7fa2f815dc30 2023-11-28T11:56:30.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f00045c0 con 0x7fa2f815dc30 2023-11-28T11:56:30.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f80b5670 con 0x7fa2f815dc30 2023-11-28T11:56:30.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0004970 con 0x7fa2f815dc30 2023-11-28T11:56:30.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.244+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f80b5390 con 0x7fa2f815dc30 2023-11-28T11:56:30.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.245+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f8160720 con 0x7fa2f815dc30 2023-11-28T11:56:30.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.245+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2f0022030 con 0x7fa2f815dc30 2023-11-28T11:56:30.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.246+0000 7fa2f4ff9700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa2e40682f0 0x7fa2e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:30.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.246+0000 7fa2f7fff700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa2e40682f0 0x7fa2e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:30.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.246+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5826+0+0 (secure 0 0 0) 0x7fa2f0086e60 con 0x7fa2f815dc30 2023-11-28T11:56:30.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.246+0000 7fa2f7fff700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa2e40682f0 0x7fa2e406a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec006fd0 tx=0x7fa2ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:30.251 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.250+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2f00537a0 con 0x7fa2f815dc30 2023-11-28T11:56:30.436 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:30.435+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2f804e620 con 0x7fa2f815dc30 2023-11-28T11:56:31.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.100+0000 7fa2f4ff9700 1 -- 172.21.15.144:0/3339581249 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa2f00170d0 con 0x7fa2f815dc30 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa2e40682f0 msgr2=0x7fa2e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa2e40682f0 0x7fa2e406a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec006fd0 tx=0x7fa2ec008040 comp rx=0 tx=0).stop 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 msgr2=0x7fa2f815f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 0x7fa2f815f7b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0015040 tx=0x7fa2f0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 shutdown_connections 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2f815dc30 0x7fa2f815f7b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa2e40682f0 0x7fa2e406a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f80cded0 0x7fa2f80cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 --2- 172.21.15.144:0/3339581249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f815a760 0x7fa2f80ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 >> 172.21.15.144:0/3339581249 conn(0x7fa2f8072620 msgr2=0x7fa2f80d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:31.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 shutdown_connections 2023-11-28T11:56:31.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.104+0000 7fa2fea1f700 1 -- 172.21.15.144:0/3339581249 wait complete. 2023-11-28T11:56:31.106 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'cephfs_data' removed 2023-11-28T11:56:31.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:56:31.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.245+0000 7f125febd700 1 Processor -- start 2023-11-28T11:56:31.246 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.245+0000 7f125febd700 1 -- start start 2023-11-28T11:56:31.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125febd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125806c990 0x7f125806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125febd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f12580ca240 0x7f12580ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125febd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125806d320 con 0x7f125806e9f0 2023-11-28T11:56:31.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125febd700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125806ae60 con 0x7f12580ca240 2023-11-28T11:56:31.248 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125febd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125806a270 con 0x7f125806c990 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f12580ca240 0x7f12580ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f12580ca240 0x7f12580ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39456/0 (socket says 172.21.15.144:39456) 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 -- 172.21.15.144:0/4257650310 learned_addr learned my addr 172.21.15.144:0/4257650310 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 -- 172.21.15.144:0/4257650310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125806c990 msgr2=0x7f125806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 --2- 172.21.15.144:0/4257650310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125806c990 0x7f125806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 -- 172.21.15.144:0/4257650310 >> v1:172.21.15.177:6789/0 conn(0x7f125806e9f0 legacy=0x7f125806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 -- 172.21.15.144:0/4257650310 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580cabe0 con 0x7f12580ca240 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125e45a700 1 --2- 172.21.15.144:0/4257650310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12580ca240 0x7f12580ca6a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f125000d150 tx=0x7f125000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686f688632b27c1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.246+0000 7f125cc57700 1 -- 172.21.15.144:0/4257650310 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250010070 con 0x7f12580ca240 2023-11-28T11:56:31.249 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125cc57700 1 -- 172.21.15.144:0/4257650310 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125000db70 con 0x7f12580ca240 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125cc57700 1 -- 172.21.15.144:0/4257650310 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250003a20 con 0x7f12580ca240 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 -- 172.21.15.144:0/4257650310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12580ca240 msgr2=0x7f12580ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 --2- 172.21.15.144:0/4257650310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12580ca240 0x7f12580ca6a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f125000d150 tx=0x7f125000d460 comp rx=0 tx=0).stop 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 -- 172.21.15.144:0/4257650310 shutdown_connections 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 --2- 172.21.15.144:0/4257650310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12580ca240 0x7f12580ca6a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 --2- 172.21.15.144:0/4257650310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f125806c990 0x7f125806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 -- 172.21.15.144:0/4257650310 >> 172.21.15.144:0/4257650310 conn(0x7f12580b12b0 msgr2=0x7f12580b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 -- 172.21.15.144:0/4257650310 shutdown_connections 2023-11-28T11:56:31.250 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.247+0000 7f125febd700 1 -- 172.21.15.144:0/4257650310 wait complete. 2023-11-28T11:56:31.253 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.248+0000 7f125febd700 1 Processor -- start 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.248+0000 7f125febd700 1 -- start start 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125febd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 0x7f12581585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125febd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125806e9f0 0x7f12580c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125febd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12580ca240 0x7f12581e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125febd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580c3000 con 0x7f125806c990 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125febd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12580ae340 con 0x7f125806e9f0 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125febd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12580c8a60 con 0x7f12580ca240 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 0x7f12581585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 0x7f12581585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39458/0 (socket says 172.21.15.144:39458) 2023-11-28T11:56:31.254 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 -- 172.21.15.144:0/2607193840 learned_addr learned my addr 172.21.15.144:0/2607193840 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125e45a700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12580ca240 0x7f12581e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125d458700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125806e9f0 0x7f12580c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 -- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12580ca240 msgr2=0x7f12581e25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12580ca240 0x7f12581e25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 -- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125806e9f0 msgr2=0x7f12580c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125806e9f0 0x7f12580c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125dc59700 1 -- 172.21.15.144:0/2607193840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125000bd40 con 0x7f125806c990 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125e45a700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12580ca240 0x7f12581e25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.249+0000 7f125d458700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125806e9f0 0x7f12580c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.250+0000 7f125dc59700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 0x7f12581585d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f125400c4a0 tx=0x7f125400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.250+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12540054e0 con 0x7f125806c990 2023-11-28T11:56:31.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.250+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1254003680 con 0x7f125806c990 2023-11-28T11:56:31.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.250+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125806a810 con 0x7f125806c990 2023-11-28T11:56:31.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.250+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254003a30 con 0x7f125806c990 2023-11-28T11:56:31.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.250+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125806a530 con 0x7f125806c990 2023-11-28T11:56:31.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.251+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125806aaf0 con 0x7f125806c990 2023-11-28T11:56:31.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.251+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1254019030 con 0x7f125806c990 2023-11-28T11:56:31.260 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.251+0000 7f124e7fc700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f123c0682f0 0x7f123c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.252+0000 7f125d458700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f123c0682f0 0x7f123c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.252+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1254084d90 con 0x7f125806c990 2023-11-28T11:56:31.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.252+0000 7f125d458700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f123c0682f0 0x7f123c06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1248007900 tx=0x7f1248008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:31.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.256+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12540518f0 con 0x7f125806c990 2023-11-28T11:56:31.439 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.438+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f12580cd5b0 con 0x7f125806c990 2023-11-28T11:56:31.443 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.442+0000 7f124e7fc700 1 -- 172.21.15.144:0/2607193840 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1254054f10 con 0x7f125806c990 2023-11-28T11:56:31.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f123c0682f0 msgr2=0x7f123c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f123c0682f0 0x7f123c06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1248007900 tx=0x7f1248008040 comp rx=0 tx=0).stop 2023-11-28T11:56:31.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 msgr2=0x7f12581585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.445 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 0x7f12581585d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f125400c4a0 tx=0x7f125400c860 comp rx=0 tx=0).stop 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 shutdown_connections 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f125806c990 0x7f12581585d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f123c0682f0 0x7f123c06a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12580ca240 0x7f12581e25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 --2- 172.21.15.144:0/2607193840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f125806e9f0 0x7f12580c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 >> 172.21.15.144:0/2607193840 conn(0x7f12580b12b0 msgr2=0x7f12580af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 shutdown_connections 2023-11-28T11:56:31.446 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.444+0000 7f125febd700 1 -- 172.21.15.144:0/2607193840 wait complete. 2023-11-28T11:56:31.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:56:31.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.579+0000 7f5db9f13700 1 Processor -- start 2023-11-28T11:56:31.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.579+0000 7f5db9f13700 1 -- start start 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db9f13700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 0x7f5db40af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db9f13700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40cc5f0 con 0x7f5db40cbdf0 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db9f13700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db415d470 con 0x7f5db406ba90 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db9f13700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db415d750 con 0x7f5db415cef0 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db2ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5db40cbdf0 0x7f5db406b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56178/0 (socket says 172.21.15.144:56178) 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db2ffd700 1 -- 172.21.15.144:0/3452395169 learned_addr learned my addr 172.21.15.144:0/3452395169 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db37fe700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 0x7f5db40af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302426430 0 0) 0x7f5db40cc5f0 con 0x7f5db40cbdf0 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da00057c0 con 0x7f5db40cbdf0 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4097621125 0 0) 0x7f5da00057c0 con 0x7f5db40cbdf0 2023-11-28T11:56:31.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 msgr2=0x7f5db40af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db27fc700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 0x7f5db40af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.580+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 >> v1:172.21.15.177:6789/0 conn(0x7f5db406ba90 legacy=0x7f5db40ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db415da30 con 0x7f5db40cbdf0 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f5da4001670 con 0x7f5db40cbdf0 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 >> v1:172.21.15.144:6789/0 conn(0x7f5db40cbdf0 legacy=0x7f5db406b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5da0006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da0006a60 0x7f5da0006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 0x7f5da0007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db415d750 con 0x7f5da0005c40 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000dcd0 con 0x7f5da0006a60 2023-11-28T11:56:31.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000de60 con 0x7f5db415cef0 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5da0006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db2ffd700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da0006a60 0x7f5da0006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db3fff700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 0x7f5da0007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 -- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 msgr2=0x7f5da0007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db415cef0 0x7f5da0007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 -- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da0006a60 msgr2=0x7f5da0006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5da0006a60 0x7f5da0006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 -- 172.21.15.144:0/3452395169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00057c0 con 0x7f5da0005c40 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db37fe700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5da0006050 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009b60 tx=0x7f5da0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94fce83c7c1b75f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00e6c0 con 0x7f5da0005c40 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c00e820 con 0x7f5da0005c40 2023-11-28T11:56:31.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.581+0000 7f5db27fc700 1 -- 172.21.15.144:0/3452395169 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00eb90 con 0x7f5da0005c40 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db9f13700 1 -- 172.21.15.144:0/3452395169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 msgr2=0x7f5da0006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db9f13700 1 --2- 172.21.15.144:0/3452395169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5da0006050 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009b60 tx=0x7f5da0007a90 comp rx=0 tx=0).stop 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db3fff700 1 -- 172.21.15.144:0/3452395169 reap_dead start 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db9f13700 1 -- 172.21.15.144:0/3452395169 shutdown_connections 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db9f13700 1 -- 172.21.15.144:0/3452395169 >> 172.21.15.144:0/3452395169 conn(0x7f5db40d0000 msgr2=0x7f5db40d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db9f13700 1 -- 172.21.15.144:0/3452395169 shutdown_connections 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.582+0000 7f5db9f13700 1 -- 172.21.15.144:0/3452395169 wait complete. 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.583+0000 7f5db9f13700 1 Processor -- start 2023-11-28T11:56:31.587 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.583+0000 7f5db9f13700 1 -- start start 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db9f13700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5db41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db9f13700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db40b3880 0x7f5db40b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db9f13700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db40b4c00 0x7f5db406dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db9f13700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b2e30 con 0x7f5da0005c40 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db9f13700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b5180 con 0x7f5db40b3880 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db9f13700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b5460 con 0x7f5db40b4c00 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5db41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5db41596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39480/0 (socket says 172.21.15.144:39480) 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 -- 172.21.15.144:0/2684445249 learned_addr learned my addr 172.21.15.144:0/2684445249 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:31.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db3fff700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db40b4c00 0x7f5db406dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db2ffd700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db40b3880 0x7f5db40b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 -- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db40b4c00 msgr2=0x7f5db406dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db40b4c00 0x7f5db406dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 -- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db40b3880 msgr2=0x7f5db40b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db40b3880 0x7f5db40b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.584+0000 7f5db37fe700 1 -- 172.21.15.144:0/2684445249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db415dff0 con 0x7f5da0005c40 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.585+0000 7f5db37fe700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5db41596c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5da00069b0 tx=0x7f5d9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.585+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00e6c0 con 0x7f5da0005c40 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.585+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c003ad0 con 0x7f5da0005c40 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.585+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c003e80 con 0x7f5da0005c40 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.585+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db415dd10 con 0x7f5da0005c40 2023-11-28T11:56:31.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.585+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db40af400 con 0x7f5da0005c40 2023-11-28T11:56:31.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.586+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db406f440 con 0x7f5da0005c40 2023-11-28T11:56:31.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.586+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d9c019030 con 0x7f5da0005c40 2023-11-28T11:56:31.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.586+0000 7f5dabfff700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d940682f0 0x7f5d9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:31.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.586+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5d9c087070 con 0x7f5da0005c40 2023-11-28T11:56:31.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.587+0000 7f5db2ffd700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d940682f0 0x7f5d9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:31.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.587+0000 7f5db2ffd700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d940682f0 0x7f5d9406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5da4001050 tx=0x7f5da4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:31.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.591+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d9c053bd0 con 0x7f5da0005c40 2023-11-28T11:56:31.851 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:31.850+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f5db40d0e60 con 0x7f5da0005c40 2023-11-28T11:56:32.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.101+0000 7f5dabfff700 1 -- 172.21.15.144:0/2684445249 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v252) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d9c0571f0 con 0x7f5da0005c40 2023-11-28T11:56:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d940682f0 msgr2=0x7f5d9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d940682f0 0x7f5d9406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5da4001050 tx=0x7f5da4009040 comp rx=0 tx=0).stop 2023-11-28T11:56:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 msgr2=0x7f5db41596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5db41596c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5da00069b0 tx=0x7f5d9c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:56:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 shutdown_connections 2023-11-28T11:56:32.105 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5da0005c40 0x7f5db41596c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5d940682f0 0x7f5d9406a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5db40b4c00 0x7f5db406dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 --2- 172.21.15.144:0/2684445249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db40b3880 0x7f5db40b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 >> 172.21.15.144:0/2684445249 conn(0x7f5db40d0000 msgr2=0x7f5db40cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 shutdown_connections 2023-11-28T11:56:32.106 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:32.103+0000 7f5db9f13700 1 -- 172.21.15.144:0/2684445249 wait complete. 2023-11-28T11:56:32.117 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T11:56:32.117 INFO:tasks.cephfs_test_runner: 2023-11-28T11:56:32.117 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:56:32.117 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:56:32.117 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=100) 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 100s and 0 retries 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner: 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner:Ran 3 tests in 287.895s 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner: 2023-11-28T11:56:32.118 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T11:56:32.120 INFO:tasks.cephfs_test_runner: 2023-11-28T11:56:32.120 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:56:32.120 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:56:32.121 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:56:32.121 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:56:32.121 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:56:32.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:56:32.121 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=100) 2023-11-28T11:56:32.122 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:56:32.122 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:56:32.122 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 100s and 0 retries 2023-11-28T11:56:32.122 INFO:tasks.cephfs_test_runner: 2023-11-28T11:56:32.122 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:56:32.197 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d52e4172190d42338d96d66aac35cf64 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_17c45c858c4cb3105f973d12c6c74f8d5037aa89/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:56:32.199 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T11:56:32.208 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T11:56:32.241 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T11:56:32.242 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:56:32.242 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:32.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:56:32.273 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T11:56:32.273 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:32.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15965 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:56:32.331 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:56:32.331 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:32.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:56:32.352 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-28T11:56:32.353 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:56:32.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17955 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:56:32.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.412 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:56:32.429 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:56:32.429 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:56:32.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:56:32.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.512 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:56:32.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.589 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-11-28T11:56:32.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.715 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-11-28T11:56:32.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.733 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-11-28T11:56:32.802 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-11-28T11:56:32.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:56:32.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.804 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T11:56:32.824 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T11:56:32.824 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:56:32.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.825 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:56:32.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:56:32.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:32.995 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T11:56:33.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:33.113 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-11-28T11:56:33.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:56:33.130 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-11-28T11:56:33.200 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-11-28T11:56:33.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:56:33.201 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T11:56:33.211 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:56:33.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:56:33.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.334+0000 7f9ee4177700 1 Processor -- start 2023-11-28T11:56:33.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.334+0000 7f9ee4177700 1 -- start start 2023-11-28T11:56:33.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.335+0000 7f9ee4177700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc159490 0x7f9edc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.335+0000 7f9ee4177700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9edc159860 0x7f9edc0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.338 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.335+0000 7f9ee4177700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9edc0cdc80 0x7f9edc0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.335+0000 7f9ee4177700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0ccc00 con 0x7f9edc159860 2023-11-28T11:56:33.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.335+0000 7f9ee4177700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0ccec0 con 0x7f9edc0cdc80 2023-11-28T11:56:33.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.335+0000 7f9ee4177700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0cd1a0 con 0x7f9edc159490 2023-11-28T11:56:33.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9edc159860 0x7f9edc0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9edc159860 0x7f9edc0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39484/0 (socket says 172.21.15.144:39484) 2023-11-28T11:56:33.339 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 -- 172.21.15.144:0/2506866103 learned_addr learned my addr 172.21.15.144:0/2506866103 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 -- 172.21.15.144:0/2506866103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc159490 msgr2=0x7f9edc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 --2- 172.21.15.144:0/2506866103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc159490 0x7f9edc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 -- 172.21.15.144:0/2506866103 >> v2:172.21.15.177:3300/0 conn(0x7f9edc0cdc80 msgr2=0x7f9edc0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 --2- 172.21.15.144:0/2506866103 >> v2:172.21.15.177:3300/0 conn(0x7f9edc0cdc80 0x7f9edc0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 -- 172.21.15.144:0/2506866103 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0cd480 con 0x7f9edc159860 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee1712700 1 --2- 172.21.15.144:0/2506866103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159860 0x7f9edc0b0bb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0b4b20 tx=0x7f9ed8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c48231936d0a0f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee0f11700 1 -- 172.21.15.144:0/2506866103 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800b870 con 0x7f9edc159860 2023-11-28T11:56:33.340 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee0f11700 1 -- 172.21.15.144:0/2506866103 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed800b9d0 con 0x7f9edc159860 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.336+0000 7f9ee0f11700 1 -- 172.21.15.144:0/2506866103 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800bd60 con 0x7f9edc159860 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 -- 172.21.15.144:0/2506866103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159860 msgr2=0x7f9edc0b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2506866103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159860 0x7f9edc0b0bb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0b4b20 tx=0x7f9ed8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 -- 172.21.15.144:0/2506866103 shutdown_connections 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2506866103 >> v2:172.21.15.177:3300/0 conn(0x7f9edc0cdc80 0x7f9edc0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2506866103 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc159490 0x7f9edc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.341 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2506866103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159860 0x7f9edc0b0bb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 -- 172.21.15.144:0/2506866103 >> 172.21.15.144:0/2506866103 conn(0x7f9edc0cb670 msgr2=0x7f9edc0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:33.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 -- 172.21.15.144:0/2506866103 shutdown_connections 2023-11-28T11:56:33.342 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.337+0000 7f9ee4177700 1 -- 172.21.15.144:0/2506866103 wait complete. 2023-11-28T11:56:33.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.338+0000 7f9ee4177700 1 Processor -- start 2023-11-28T11:56:33.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.338+0000 7f9ee4177700 1 -- start start 2023-11-28T11:56:33.345 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9edc0cdc80 0x7f9edc0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 0x7f9edc0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc0c4b10 0x7f9edc0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0c8c60 con 0x7f9edc159490 2023-11-28T11:56:33.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc152a70 con 0x7f9edc0cdc80 2023-11-28T11:56:33.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0c86a0 con 0x7f9edc0c4b10 2023-11-28T11:56:33.346 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 0x7f9edc0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 0x7f9edc0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39492/0 (socket says 172.21.15.144:39492) 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 -- 172.21.15.144:0/2508014163 learned_addr learned my addr 172.21.15.144:0/2508014163 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1f13700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9edc0cdc80 0x7f9edc0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee2714700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc0c4b10 0x7f9edc0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 -- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc0c4b10 msgr2=0x7f9edc0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc0c4b10 0x7f9edc0b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.347 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 -- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9edc0cdc80 msgr2=0x7f9edc0c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9edc0cdc80 0x7f9edc0c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 -- 172.21.15.144:0/2508014163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009550 con 0x7f9edc159490 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1f13700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9edc0cdc80 0x7f9edc0c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee2714700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc0c4b10 0x7f9edc0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee1712700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 0x7f9edc0b1a20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8014040 tx=0x7f9ed8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8004520 con 0x7f9edc159490 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed801e070 con 0x7f9edc159490 2023-11-28T11:56:33.348 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80038c0 con 0x7f9edc159490 2023-11-28T11:56:33.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.339+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc0c9a00 con 0x7f9edc159490 2023-11-28T11:56:33.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.340+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc0cd760 con 0x7f9edc159490 2023-11-28T11:56:33.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.340+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc0c9b90 con 0x7f9edc159490 2023-11-28T11:56:33.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.341+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ed800f050 con 0x7f9edc159490 2023-11-28T11:56:33.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.341+0000 7f9ed27fc700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ec00682f0 0x7f9ec006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.349 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.341+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9ed8020070 con 0x7f9edc159490 2023-11-28T11:56:33.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.341+0000 7f9ee1f13700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ec00682f0 0x7f9ec006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.342+0000 7f9ee1f13700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ec00682f0 0x7f9ec006a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0099d0 tx=0x7f9ecc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:33.350 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.345+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ed8053c30 con 0x7f9edc159490 2023-11-28T11:56:33.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.523+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9edc0745c0 con 0x7f9ec00682f0 2023-11-28T11:56:33.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.525+0000 7f9ed27fc700 1 -- 172.21.15.144:0/2508014163 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73267 (secure 0 0 0) 0x7f9edc0745c0 con 0x7f9ec00682f0 2023-11-28T11:56:33.527 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:33.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.527+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ec00682f0 msgr2=0x7f9ec006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ec00682f0 0x7f9ec006a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0099d0 tx=0x7f9ecc008040 comp rx=0 tx=0).stop 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 msgr2=0x7f9edc0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 0x7f9edc0b1a20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8014040 tx=0x7f9ed8004750 comp rx=0 tx=0).stop 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 shutdown_connections 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9ec00682f0 0x7f9ec006a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9edc0c4b10 0x7f9edc0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9edc0cdc80 0x7f9edc0c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 --2- 172.21.15.144:0/2508014163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc159490 0x7f9edc0b1a20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 >> 172.21.15.144:0/2508014163 conn(0x7f9edc0cb670 msgr2=0x7f9edc0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:33.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 shutdown_connections 2023-11-28T11:56:33.530 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.528+0000 7f9ee4177700 1 -- 172.21.15.144:0/2508014163 wait complete. 2023-11-28T11:56:33.530 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:56:33.540 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-11-28T11:56:32.557553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":83396,"kb_used_data":7108,"kb_used_omap":0,"kb_used_meta":76224,"kb_avail":754891324,"statfs":{"total":773094113280,"available":773008715776,"internally_reserved":0,"allocated":7278592,"data_stored":3868364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78053376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-4,"num_write":-77,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":0,"num_keys_recovered":-1,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001430"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.742270+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:55:24.742270+0000","last_peered":"2023-11-28T11:55:24.742270+0000","last_clean":"2023-11-28T11:55:24.742270+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:55:24.742270+0000","last_undegraded":"2023-11-28T11:55:24.742270+0000","last_fullsized":"2023-11-28T11:55:24.742270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.736439+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:55:24.736439+0000","last_peered":"2023-11-28T11:55:24.736439+0000","last_clean":"2023-11-28T11:55:24.736439+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:55:24.736439+0000","last_undegraded":"2023-11-28T11:55:24.736439+0000","last_fullsized":"2023-11-28T11:55:24.736439+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.738956+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:55:24.738956+0000","last_peered":"2023-11-28T11:55:24.738956+0000","last_clean":"2023-11-28T11:55:24.738956+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:55:24.738956+0000","last_undegraded":"2023-11-28T11:55:24.738956+0000","last_fullsized":"2023-11-28T11:55:24.738956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:14.749644+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:55:14.749644+0000","last_peered":"2023-11-28T11:55:14.749644+0000","last_clean":"2023-11-28T11:55:14.749644+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:55:14.749644+0000","last_undegraded":"2023-11-28T11:55:14.749644+0000","last_fullsized":"2023-11-28T11:55:14.749644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.751238+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:55:24.751238+0000","last_peered":"2023-11-28T11:55:24.751238+0000","last_clean":"2023-11-28T11:55:24.751238+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:55:24.751238+0000","last_undegraded":"2023-11-28T11:55:24.751238+0000","last_fullsized":"2023-11-28T11:55:24.751238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:14.749695+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:55:14.749695+0000","last_peered":"2023-11-28T11:55:14.749695+0000","last_clean":"2023-11-28T11:55:14.749695+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:55:14.749695+0000","last_undegraded":"2023-11-28T11:55:14.749695+0000","last_fullsized":"2023-11-28T11:55:14.749695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.742490+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:55:24.742490+0000","last_peered":"2023-11-28T11:55:24.742490+0000","last_clean":"2023-11-28T11:55:24.742490+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:55:24.742490+0000","last_undegraded":"2023-11-28T11:55:24.742490+0000","last_fullsized":"2023-11-28T11:55:24.742490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:34.743656+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:55:34.743656+0000","last_peered":"2023-11-28T11:55:34.743656+0000","last_clean":"2023-11-28T11:55:34.743656+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:55:34.743656+0000","last_undegraded":"2023-11-28T11:55:34.743656+0000","last_fullsized":"2023-11-28T11:55:34.743656+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:34.743705+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:55:34.743705+0000","last_peered":"2023-11-28T11:55:34.743705+0000","last_clean":"2023-11-28T11:55:34.743705+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:55:34.743705+0000","last_undegraded":"2023-11-28T11:55:34.743705+0000","last_fullsized":"2023-11-28T11:55:34.743705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542212,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10864,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360976,"statfs":{"total":96636764160,"available":96625639424,"internally_reserved":0,"allocated":1351680,"data_stored":918302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70799999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59399999999999997,"5min":0.751,"15min":0.751},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.55600000000000005},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67000000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64400000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49299999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542211,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.81599999999999995,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51500000000000001,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.37}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.56899999999999995},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81299999999999994,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59099999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10704,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361136,"statfs":{"total":96636764160,"available":96625803264,"internally_reserved":0,"allocated":1384448,"data_stored":936802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32300000000000001,"15min":0.32800000000000001},"min":{"1min":0.224,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.43099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34899999999999998},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53200000000000003,"5min":0.622,"15min":0.622},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.439},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86199999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63100000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66200000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542212,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13668,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358172,"statfs":{"total":96636764160,"available":96622768128,"internally_reserved":0,"allocated":749568,"data_stored":332912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.32300000000000001},"min":{"1min":0.23599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47699999999999998,"5min":0.754,"15min":0.754},"last":0.254},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35899999999999999,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.219}]},{"osd":2,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.498,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.317},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56299999999999994,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.28899999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71399999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":0.87,"15min":0.87},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.46600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70799999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9512,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362328,"statfs":{"total":96636764160,"available":96627023872,"internally_reserved":0,"allocated":753664,"data_stored":333898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.45200000000000001,"15min":0.64400000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.499,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.32100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47699999999999998,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.51100000000000001,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.371}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.34699999999999998},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.53100000000000003,"5min":0.622,"15min":0.622},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49199999999999999,"15min":0.65800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54800000000000004,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69099999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.89300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":8.359,"15min":8.359},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.745},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68799999999999994,"15min":0.83999999999999997},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88300000000000001,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.69799999999999995,"15min":0.878},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86099999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.90600000000000003},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.75},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81200000000000006,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.66900000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542212,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94362008,"statfs":{"total":96636764160,"available":96626696192,"internally_reserved":0,"allocated":753664,"data_stored":333898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":1.762,"15min":1.762},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.27600000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.51900000000000002,"15min":0.91800000000000004},"min":{"1min":0.253,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52500000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55800000000000005,"5min":1.857,"15min":1.857},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.50800000000000001,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.59399999999999997,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0249999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.96799999999999997,"5min":1.498,"15min":1.498},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93799999999999994,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.501}]},{"osd":6,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61599999999999999,"15min":0.56999999999999995},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.48199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96499999999999997,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.441}]},{"osd":7,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54600000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542212,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":774144,"data_stored":345686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.248,"15min":0.248},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.749},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":2.137,"15min":2.137},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71599999999999997,"5min":0.751,"15min":0.751},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.45400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.48399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.745,"15min":0.745},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67700000000000005,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.34899999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63800000000000001,"5min":2.286,"15min":2.286},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.63300000000000001},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59399999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9576,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362264,"statfs":{"total":96636764160,"available":96626958336,"internally_reserved":0,"allocated":753664,"data_stored":332934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.52100000000000002},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"min":{"1min":0.48299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.42499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.53200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.497,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.679,"15min":1.679},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.749,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:56:33.543 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:56:33.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.661+0000 7fd1b736a700 1 Processor -- start 2023-11-28T11:56:33.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.662+0000 7fd1b736a700 1 -- start start 2023-11-28T11:56:33.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.662+0000 7fd1b736a700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1b015c9b0 0x7fd1b00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b736a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd1b0161340 0x7fd1b00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b736a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00d1950 con 0x7fd1b00d2320 2023-11-28T11:56:33.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b736a700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b00b5800 con 0x7fd1b015c9b0 2023-11-28T11:56:33.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b736a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b00b5520 con 0x7fd1b0161340 2023-11-28T11:56:33.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5106700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1b015c9b0 0x7fd1b00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5907700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd1b00d2320 0x7fd1b00cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56214/0 (socket says 172.21.15.144:56214) 2023-11-28T11:56:33.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5907700 1 -- 172.21.15.144:0/3613153927 learned_addr learned my addr 172.21.15.144:0/3613153927 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:33.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b4905700 1 --2- 172.21.15.144:0/3613153927 >> v2:172.21.15.177:3300/0 conn(0x7fd1b0161340 0x7fd1b00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5106700 1 -- 172.21.15.144:0/3613153927 >> v2:172.21.15.177:3300/0 conn(0x7fd1b0161340 msgr2=0x7fd1b00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:33.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5106700 1 --2- 172.21.15.144:0/3613153927 >> v2:172.21.15.177:3300/0 conn(0x7fd1b0161340 0x7fd1b00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5106700 1 -- 172.21.15.144:0/3613153927 >> v1:172.21.15.144:6789/0 conn(0x7fd1b00d2320 legacy=0x7fd1b00cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5106700 1 -- 172.21.15.144:0/3613153927 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b00d1300 con 0x7fd1b015c9b0 2023-11-28T11:56:33.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1a7fff700 1 -- 172.21.15.144:0/3613153927 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647365284 0 0) 0x7fd1b00d1950 con 0x7fd1b00d2320 2023-11-28T11:56:33.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1b5106700 1 --2- 172.21.15.144:0/3613153927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b00b4bf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00685c0 tx=0x7fd1a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4d9dfaf04af184 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:33.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1a7fff700 1 -- 172.21.15.144:0/3613153927 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0011830 con 0x7fd1b015c9b0 2023-11-28T11:56:33.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1a7fff700 1 -- 172.21.15.144:0/3613153927 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a0011990 con 0x7fd1b015c9b0 2023-11-28T11:56:33.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.663+0000 7fd1a7fff700 1 -- 172.21.15.144:0/3613153927 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0011d20 con 0x7fd1b015c9b0 2023-11-28T11:56:33.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 -- 172.21.15.144:0/3613153927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 msgr2=0x7fd1b00b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 --2- 172.21.15.144:0/3613153927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b00b4bf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00685c0 tx=0x7fd1a0009b20 comp rx=0 tx=0).stop 2023-11-28T11:56:33.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 -- 172.21.15.144:0/3613153927 shutdown_connections 2023-11-28T11:56:33.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 --2- 172.21.15.144:0/3613153927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b00b4bf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 --2- 172.21.15.144:0/3613153927 >> v2:172.21.15.177:3300/0 conn(0x7fd1b0161340 0x7fd1b00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 -- 172.21.15.144:0/3613153927 >> 172.21.15.144:0/3613153927 conn(0x7fd1b0072600 msgr2=0x7fd1b00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:33.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 -- 172.21.15.144:0/3613153927 shutdown_connections 2023-11-28T11:56:33.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.664+0000 7fd1b736a700 1 -- 172.21.15.144:0/3613153927 wait complete. 2023-11-28T11:56:33.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.666+0000 7fd1b736a700 1 Processor -- start 2023-11-28T11:56:33.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.666+0000 7fd1b736a700 1 -- start start 2023-11-28T11:56:33.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.666+0000 7fd1b736a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b00d2320 0x7fd1b0150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b736a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b0159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b736a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0161340 0x7fd1b01599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b736a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01e88a0 con 0x7fd1b015c9b0 2023-11-28T11:56:33.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b736a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00b7d70 con 0x7fd1b00d2320 2023-11-28T11:56:33.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b736a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01f70c0 con 0x7fd1b0161340 2023-11-28T11:56:33.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b0159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b0159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39504/0 (socket says 172.21.15.144:39504) 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 -- 172.21.15.144:0/2423785930 learned_addr learned my addr 172.21.15.144:0/2423785930 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b5106700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b00d2320 0x7fd1b0150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 -- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0161340 msgr2=0x7fd1b01599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b5907700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0161340 0x7fd1b01599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0161340 0x7fd1b01599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 -- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b00d2320 msgr2=0x7fd1b0150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b00d2320 0x7fd1b0150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 -- 172.21.15.144:0/2423785930 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a00094a0 con 0x7fd1b015c9b0 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b5106700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b00d2320 0x7fd1b0150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b5907700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0161340 0x7fd1b01599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b4905700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b0159480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00e9b0 tx=0x7fd1ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac009710 con 0x7fd1b015c9b0 2023-11-28T11:56:33.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac00c850 con 0x7fd1b015c9b0 2023-11-28T11:56:33.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b00b5130 con 0x7fd1b015c9b0 2023-11-28T11:56:33.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.667+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00cbc0 con 0x7fd1b015c9b0 2023-11-28T11:56:33.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.668+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b00d15e0 con 0x7fd1b015c9b0 2023-11-28T11:56:33.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.668+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b00b52c0 con 0x7fd1b015c9b0 2023-11-28T11:56:33.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.669+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd1ac019030 con 0x7fd1b015c9b0 2023-11-28T11:56:33.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.669+0000 7fd1a5ffb700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd1940682f0 0x7fd19406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.669+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd1ac085a70 con 0x7fd1b015c9b0 2023-11-28T11:56:33.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.669+0000 7fd1b5106700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd1940682f0 0x7fd19406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.670+0000 7fd1b5106700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd1940682f0 0x7fd19406a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00090b0 tx=0x7fd1a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:33.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.673+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd1ac052500 con 0x7fd1b015c9b0 2023-11-28T11:56:33.854 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.854+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1b01e89e0 con 0x7fd1940682f0 2023-11-28T11:56:33.857 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.856+0000 7fd1a5ffb700 1 -- 172.21.15.144:0/2423785930 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73267 (secure 0 0 0) 0x7fd1b01e89e0 con 0x7fd1940682f0 2023-11-28T11:56:33.860 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.858+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd1940682f0 msgr2=0x7fd19406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd1940682f0 0x7fd19406a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00090b0 tx=0x7fd1a0008030 comp rx=0 tx=0).stop 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 msgr2=0x7fd1b0159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b0159480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00e9b0 tx=0x7fd1ac00ed70 comp rx=0 tx=0).stop 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 shutdown_connections 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd1940682f0 0x7fd19406a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd1b0161340 0x7fd1b01599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1b00d2320 0x7fd1b0150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 --2- 172.21.15.144:0/2423785930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b015c9b0 0x7fd1b0159480 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 >> 172.21.15.144:0/2423785930 conn(0x7fd1b0072600 msgr2=0x7fd1b015fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 shutdown_connections 2023-11-28T11:56:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.859+0000 7fd1b736a700 1 -- 172.21.15.144:0/2423785930 wait complete. 2023-11-28T11:56:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:56:33.871 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-11-28T11:56:32.557553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":83396,"kb_used_data":7108,"kb_used_omap":0,"kb_used_meta":76224,"kb_avail":754891324,"statfs":{"total":773094113280,"available":773008715776,"internally_reserved":0,"allocated":7278592,"data_stored":3868364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78053376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-4,"num_write":-77,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":0,"num_keys_recovered":-1,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001430"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.742270+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:55:24.742270+0000","last_peered":"2023-11-28T11:55:24.742270+0000","last_clean":"2023-11-28T11:55:24.742270+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:55:24.742270+0000","last_undegraded":"2023-11-28T11:55:24.742270+0000","last_fullsized":"2023-11-28T11:55:24.742270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.736439+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:55:24.736439+0000","last_peered":"2023-11-28T11:55:24.736439+0000","last_clean":"2023-11-28T11:55:24.736439+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:55:24.736439+0000","last_undegraded":"2023-11-28T11:55:24.736439+0000","last_fullsized":"2023-11-28T11:55:24.736439+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.738956+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:55:24.738956+0000","last_peered":"2023-11-28T11:55:24.738956+0000","last_clean":"2023-11-28T11:55:24.738956+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:55:24.738956+0000","last_undegraded":"2023-11-28T11:55:24.738956+0000","last_fullsized":"2023-11-28T11:55:24.738956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:14.749644+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:55:14.749644+0000","last_peered":"2023-11-28T11:55:14.749644+0000","last_clean":"2023-11-28T11:55:14.749644+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:55:14.749644+0000","last_undegraded":"2023-11-28T11:55:14.749644+0000","last_fullsized":"2023-11-28T11:55:14.749644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.751238+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:55:24.751238+0000","last_peered":"2023-11-28T11:55:24.751238+0000","last_clean":"2023-11-28T11:55:24.751238+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:55:24.751238+0000","last_undegraded":"2023-11-28T11:55:24.751238+0000","last_fullsized":"2023-11-28T11:55:24.751238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:14.749695+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:55:14.749695+0000","last_peered":"2023-11-28T11:55:14.749695+0000","last_clean":"2023-11-28T11:55:14.749695+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:55:14.749695+0000","last_undegraded":"2023-11-28T11:55:14.749695+0000","last_fullsized":"2023-11-28T11:55:14.749695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.742490+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:55:24.742490+0000","last_peered":"2023-11-28T11:55:24.742490+0000","last_clean":"2023-11-28T11:55:24.742490+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:55:24.742490+0000","last_undegraded":"2023-11-28T11:55:24.742490+0000","last_fullsized":"2023-11-28T11:55:24.742490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:34.743656+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:55:34.743656+0000","last_peered":"2023-11-28T11:55:34.743656+0000","last_clean":"2023-11-28T11:55:34.743656+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:55:34.743656+0000","last_undegraded":"2023-11-28T11:55:34.743656+0000","last_fullsized":"2023-11-28T11:55:34.743656+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:34.743705+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:55:34.743705+0000","last_peered":"2023-11-28T11:55:34.743705+0000","last_clean":"2023-11-28T11:55:34.743705+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:55:34.743705+0000","last_undegraded":"2023-11-28T11:55:34.743705+0000","last_fullsized":"2023-11-28T11:55:34.743705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542212,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10864,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360976,"statfs":{"total":96636764160,"available":96625639424,"internally_reserved":0,"allocated":1351680,"data_stored":918302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70799999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59399999999999997,"5min":0.751,"15min":0.751},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.55600000000000005},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67000000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64400000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49299999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542211,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.81599999999999995,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51500000000000001,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.37}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.56899999999999995},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81299999999999994,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59099999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10704,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361136,"statfs":{"total":96636764160,"available":96625803264,"internally_reserved":0,"allocated":1384448,"data_stored":936802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32300000000000001,"15min":0.32800000000000001},"min":{"1min":0.224,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.43099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34899999999999998},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53200000000000003,"5min":0.622,"15min":0.622},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.439},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86199999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63100000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66200000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542212,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13668,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358172,"statfs":{"total":96636764160,"available":96622768128,"internally_reserved":0,"allocated":749568,"data_stored":332912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.32300000000000001},"min":{"1min":0.23599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47699999999999998,"5min":0.754,"15min":0.754},"last":0.254},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35899999999999999,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.219}]},{"osd":2,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.498,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.317},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56299999999999994,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.28899999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71399999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":0.87,"15min":0.87},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.46600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70799999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9512,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362328,"statfs":{"total":96636764160,"available":96627023872,"internally_reserved":0,"allocated":753664,"data_stored":333898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.45200000000000001,"15min":0.64400000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.499,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.32100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47699999999999998,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.51100000000000001,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.371}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.34699999999999998},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.53100000000000003,"5min":0.622,"15min":0.622},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49199999999999999,"15min":0.65800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54800000000000004,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69099999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.89300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":8.359,"15min":8.359},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.745},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68799999999999994,"15min":0.83999999999999997},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88300000000000001,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.69799999999999995,"15min":0.878},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86099999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.90600000000000003},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.75},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81200000000000006,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.66900000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542212,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94362008,"statfs":{"total":96636764160,"available":96626696192,"internally_reserved":0,"allocated":753664,"data_stored":333898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":1.762,"15min":1.762},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.27600000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.51900000000000002,"15min":0.91800000000000004},"min":{"1min":0.253,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52500000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55800000000000005,"5min":1.857,"15min":1.857},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.50800000000000001,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.59399999999999997,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0249999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.96799999999999997,"5min":1.498,"15min":1.498},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93799999999999994,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.501}]},{"osd":6,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61599999999999999,"15min":0.56999999999999995},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.48199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96499999999999997,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.441}]},{"osd":7,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54600000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542212,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":774144,"data_stored":345686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.248,"15min":0.248},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.749},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":2.137,"15min":2.137},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71599999999999997,"5min":0.751,"15min":0.751},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.45400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.48399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.745,"15min":0.745},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67700000000000005,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.34899999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63800000000000001,"5min":2.286,"15min":2.286},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.63300000000000001},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59399999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9576,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362264,"statfs":{"total":96636764160,"available":96626958336,"internally_reserved":0,"allocated":753664,"data_stored":332934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.52100000000000002},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"min":{"1min":0.48299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.42499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.53200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.497,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.679,"15min":1.679},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.749,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:56:33.873 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:56:33.873 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:56:33.993 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 Processor -- start 2023-11-28T11:56:33.994 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 -- start start 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f850c0ce040 0x7f850c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f850c0b2930 0x7f850c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c06b420 con 0x7f850c0d0a00 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c0b32f0 con 0x7f850c0ce040 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.993+0000 7f85131db700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c06ff60 con 0x7f850c0b2930 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8510f77700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f850c0ce040 0x7f850c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.997 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f850bfff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f850c0d0a00 0x7f850c06f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56224/0 (socket says 172.21.15.144:56224) 2023-11-28T11:56:33.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f850bfff700 1 -- 172.21.15.144:0/797525992 learned_addr learned my addr 172.21.15.144:0/797525992 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:33.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8511778700 1 --2- 172.21.15.144:0/797525992 >> v2:172.21.15.177:3300/0 conn(0x7f850c0b2930 0x7f850c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:33.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8510f77700 1 -- 172.21.15.144:0/797525992 >> v2:172.21.15.177:3300/0 conn(0x7f850c0b2930 msgr2=0x7f850c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:33.998 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8510f77700 1 --2- 172.21.15.144:0/797525992 >> v2:172.21.15.177:3300/0 conn(0x7f850c0b2930 0x7f850c0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8510f77700 1 -- 172.21.15.144:0/797525992 >> v1:172.21.15.144:6789/0 conn(0x7f850c0d0a00 legacy=0x7f850c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8510f77700 1 -- 172.21.15.144:0/797525992 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c06b700 con 0x7f850c0ce040 2023-11-28T11:56:34.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f850b7fe700 1 -- 172.21.15.144:0/797525992 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749883144 0 0) 0x7f850c06b420 con 0x7f850c0d0a00 2023-11-28T11:56:34.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f8510f77700 1 --2- 172.21.15.144:0/797525992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0ce040 0x7f850c15df10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f850c073df0 tx=0x7f8500009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f347370443c1007b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:34.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f850b7fe700 1 -- 172.21.15.144:0/797525992 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500005e30 con 0x7f850c0ce040 2023-11-28T11:56:34.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f850b7fe700 1 -- 172.21.15.144:0/797525992 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500006030 con 0x7f850c0ce040 2023-11-28T11:56:34.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.994+0000 7f850b7fe700 1 -- 172.21.15.144:0/797525992 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000063c0 con 0x7f850c0ce040 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 -- 172.21.15.144:0/797525992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0ce040 msgr2=0x7f850c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 --2- 172.21.15.144:0/797525992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0ce040 0x7f850c15df10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f850c073df0 tx=0x7f8500009b00 comp rx=0 tx=0).stop 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 -- 172.21.15.144:0/797525992 shutdown_connections 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 --2- 172.21.15.144:0/797525992 >> v2:172.21.15.177:3300/0 conn(0x7f850c0b2930 0x7f850c0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 --2- 172.21.15.144:0/797525992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0ce040 0x7f850c15df10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 -- 172.21.15.144:0/797525992 >> 172.21.15.144:0/797525992 conn(0x7f850c06aed0 msgr2=0x7f850c06df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 -- 172.21.15.144:0/797525992 shutdown_connections 2023-11-28T11:56:34.005 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.995+0000 7f85131db700 1 -- 172.21.15.144:0/797525992 wait complete. 2023-11-28T11:56:34.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.996+0000 7f85131db700 1 Processor -- start 2023-11-28T11:56:34.006 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.996+0000 7f85131db700 1 -- start start 2023-11-28T11:56:34.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f85131db700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f850c0b2930 0x7f850c0c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f85131db700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f850c0ce040 0x7f850c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f85131db700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 0x7f850c14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f85131db700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0c4c30 con 0x7f850c0d0a00 2023-11-28T11:56:34.009 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f85131db700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1f5730 con 0x7f850c0b2930 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f85131db700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1f59a0 con 0x7f850c0ce040 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 0x7f850c14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 0x7f850c14cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39520/0 (socket says 172.21.15.144:39520) 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 -- 172.21.15.144:0/1359062627 learned_addr learned my addr 172.21.15.144:0/1359062627 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8510f77700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f850c0b2930 0x7f850c0c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f850bfff700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f850c0ce040 0x7f850c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.010 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 -- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f850c0ce040 msgr2=0x7f850c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f850c0ce040 0x7f850c0cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 -- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f850c0b2930 msgr2=0x7f850c0c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f850c0b2930 0x7f850c0c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 -- 172.21.15.144:0/1359062627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500009430 con 0x7f850c0d0a00 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.997+0000 7f8511778700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 0x7f850c14cf00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8504007e90 tx=0x7f850400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.998+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400ed30 con 0x7f850c0d0a00 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.998+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850400ee90 con 0x7f850c0d0a00 2023-11-28T11:56:34.011 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.998+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c06bcc0 con 0x7f850c0d0a00 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.998+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504014660 con 0x7f850c0d0a00 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.998+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c06b9e0 con 0x7f850c0d0a00 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.999+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c06bfa0 con 0x7f850c0d0a00 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.999+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8504012050 con 0x7f850c0d0a00 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.999+0000 7f85097fa700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f84f00682f0 0x7f84f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.999+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8504085e80 con 0x7f850c0d0a00 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:33.999+0000 7f8510f77700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f84f00682f0 0x7f84f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.012 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.000+0000 7f8510f77700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f84f00682f0 0x7f84f006a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8500000c00 tx=0x7f8500008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:34.013 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.004+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8504052980 con 0x7f850c0d0a00 2023-11-28T11:56:34.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.182+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f850c1f5b60 con 0x7f84f00682f0 2023-11-28T11:56:34.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.185+0000 7f85097fa700 1 -- 172.21.15.144:0/1359062627 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73267 (secure 0 0 0) 0x7f8500005d30 con 0x7f84f00682f0 2023-11-28T11:56:34.186 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:34.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f84f00682f0 msgr2=0x7f84f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f84f00682f0 0x7f84f006a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8500000c00 tx=0x7f8500008070 comp rx=0 tx=0).stop 2023-11-28T11:56:34.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 msgr2=0x7f850c14cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 0x7f850c14cf00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8504007e90 tx=0x7f850400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 shutdown_connections 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f850c0d0a00 0x7f850c14cf00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f84f00682f0 0x7f84f006a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f850c0b2930 0x7f850c0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 --2- 172.21.15.144:0/1359062627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f850c0ce040 0x7f850c0cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 >> 172.21.15.144:0/1359062627 conn(0x7f850c06aed0 msgr2=0x7f850c15cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:34.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 shutdown_connections 2023-11-28T11:56:34.190 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.187+0000 7f85131db700 1 -- 172.21.15.144:0/1359062627 wait complete. 2023-11-28T11:56:34.190 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:56:34.199 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-11-28T11:56:32.557553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":83396,"kb_used_data":7108,"kb_used_omap":0,"kb_used_meta":76224,"kb_avail":754891324,"statfs":{"total":773094113280,"available":773008715776,"internally_reserved":0,"allocated":7278592,"data_stored":3868364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78053376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12570,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-5,"num_read_kb":-4,"num_write":-77,"num_write_kb":-38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":0,"num_keys_recovered":-1,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001430"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.742270+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:55:24.742270+0000","last_peered":"2023-11-28T11:55:24.742270+0000","last_clean":"2023-11-28T11:55:24.742270+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:55:24.742270+0000","last_undegraded":"2023-11-28T11:55:24.742270+0000","last_fullsized":"2023-11-28T11:55:24.742270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.736439+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:55:24.736439+0000","last_peered":"2023-11-28T11:55:24.736439+0000","last_clean":"2023-11-28T11:55:24.736439+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:55:24.736439+0000","last_undegraded":"2023-11-28T11:55:24.736439+0000","last_fullsized":"2023-11-28T11:55:24.736439+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.738956+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:55:24.738956+0000","last_peered":"2023-11-28T11:55:24.738956+0000","last_clean":"2023-11-28T11:55:24.738956+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:55:24.738956+0000","last_undegraded":"2023-11-28T11:55:24.738956+0000","last_fullsized":"2023-11-28T11:55:24.738956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:14.749644+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:55:14.749644+0000","last_peered":"2023-11-28T11:55:14.749644+0000","last_clean":"2023-11-28T11:55:14.749644+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:55:14.749644+0000","last_undegraded":"2023-11-28T11:55:14.749644+0000","last_fullsized":"2023-11-28T11:55:14.749644+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.751238+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:55:24.751238+0000","last_peered":"2023-11-28T11:55:24.751238+0000","last_clean":"2023-11-28T11:55:24.751238+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:55:24.751238+0000","last_undegraded":"2023-11-28T11:55:24.751238+0000","last_fullsized":"2023-11-28T11:55:24.751238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:14.749695+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:55:14.749695+0000","last_peered":"2023-11-28T11:55:14.749695+0000","last_clean":"2023-11-28T11:55:14.749695+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:55:14.749695+0000","last_undegraded":"2023-11-28T11:55:14.749695+0000","last_fullsized":"2023-11-28T11:55:14.749695+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:24.742490+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:55:24.742490+0000","last_peered":"2023-11-28T11:55:24.742490+0000","last_clean":"2023-11-28T11:55:24.742490+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:55:24.742490+0000","last_undegraded":"2023-11-28T11:55:24.742490+0000","last_fullsized":"2023-11-28T11:55:24.742490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:34.743656+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:55:34.743656+0000","last_peered":"2023-11-28T11:55:34.743656+0000","last_clean":"2023-11-28T11:55:34.743656+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:55:34.743656+0000","last_undegraded":"2023-11-28T11:55:34.743656+0000","last_fullsized":"2023-11-28T11:55:34.743656+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:55:34.743705+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:55:34.743705+0000","last_peered":"2023-11-28T11:55:34.743705+0000","last_clean":"2023-11-28T11:55:34.743705+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:55:34.743705+0000","last_undegraded":"2023-11-28T11:55:34.743705+0000","last_fullsized":"2023-11-28T11:55:34.743705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542212,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10864,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360976,"statfs":{"total":96636764160,"available":96625639424,"internally_reserved":0,"allocated":1351680,"data_stored":918302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70799999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.433}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59399999999999997,"5min":0.751,"15min":0.751},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.55600000000000005},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67000000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64400000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49299999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542211,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9452,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362388,"statfs":{"total":96636764160,"available":96627085312,"internally_reserved":0,"allocated":757760,"data_stored":333932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55400000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.81599999999999995,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51500000000000001,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.37}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.56899999999999995},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.45800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81299999999999994,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59099999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10704,"kb_used_data":1352,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361136,"statfs":{"total":96636764160,"available":96625803264,"internally_reserved":0,"allocated":1384448,"data_stored":936802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32300000000000001,"15min":0.32800000000000001},"min":{"1min":0.224,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.43099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34899999999999998},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53200000000000003,"5min":0.622,"15min":0.622},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.439},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86199999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63100000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66200000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542212,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13668,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":12928,"kb_avail":94358172,"statfs":{"total":96636764160,"available":96622768128,"internally_reserved":0,"allocated":749568,"data_stored":332912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13238272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.32300000000000001},"min":{"1min":0.23599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47699999999999998,"5min":0.754,"15min":0.754},"last":0.254},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35899999999999999,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.219}]},{"osd":2,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34399999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.498,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.317},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56299999999999994,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.28899999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71399999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":0.87,"15min":0.87},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.46600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70799999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9512,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362328,"statfs":{"total":96636764160,"available":96627023872,"internally_reserved":0,"allocated":753664,"data_stored":333898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.45200000000000001,"15min":0.64400000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.499,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.32100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47699999999999998,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.51100000000000001,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.371}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.34699999999999998},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.53100000000000003,"5min":0.622,"15min":0.622},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49199999999999999,"15min":0.65800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54800000000000004,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69099999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.89300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":8.359,"15min":8.359},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.745},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68799999999999994,"15min":0.83999999999999997},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88300000000000001,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.69799999999999995,"15min":0.878},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86099999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.90600000000000003},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.75},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81200000000000006,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.66900000000000004}]}]},{"osd":3,"up_from":14,"seq":60129542212,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9832,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94362008,"statfs":{"total":96636764160,"available":96626696192,"internally_reserved":0,"allocated":753664,"data_stored":333898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":1.762,"15min":1.762},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.27600000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.51900000000000002,"15min":0.91800000000000004},"min":{"1min":0.253,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52500000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55800000000000005,"5min":1.857,"15min":1.857},"last":0.30499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.50800000000000001,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.59399999999999997,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0249999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.96799999999999997,"5min":1.498,"15min":1.498},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93799999999999994,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.501}]},{"osd":6,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61599999999999999,"15min":0.56999999999999995},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.48199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96499999999999997,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.441}]},{"osd":7,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54600000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542212,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":774144,"data_stored":345686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.248,"15min":0.248},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.749},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":2.137,"15min":2.137},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71599999999999997,"5min":0.751,"15min":0.751},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.45400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.48399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.745,"15min":0.745},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67700000000000005,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.34899999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63800000000000001,"5min":2.286,"15min":2.286},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.63300000000000001},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59399999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542212,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9576,"kb_used_data":736,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362264,"statfs":{"total":96636764160,"available":96626958336,"internally_reserved":0,"allocated":753664,"data_stored":332934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.52100000000000002},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"min":{"1min":0.48299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.42499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.53200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.497,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.679,"15min":1.679},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.749,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:56:34.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:56:34.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.321+0000 7f4168557700 1 Processor -- start 2023-11-28T11:56:34.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.321+0000 7f4168557700 1 -- start start 2023-11-28T11:56:34.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.322+0000 7f4168557700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4160071040 0x7f4160071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.322+0000 7f4168557700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4160071950 con 0x7f416006dee0 2023-11-28T11:56:34.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.322+0000 7f4168557700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416015d060 con 0x7f4160160540 2023-11-28T11:56:34.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.322+0000 7f4168557700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416015d340 con 0x7f4160071040 2023-11-28T11:56:34.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f4165af2700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f416006dee0 0x7f416006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56238/0 (socket says 172.21.15.144:56238) 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f4165af2700 1 -- 172.21.15.144:0/1793476523 learned_addr learned my addr 172.21.15.144:0/1793476523 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41662f3700 1 --2- 172.21.15.144:0/1793476523 >> v2:172.21.15.177:3300/0 conn(0x7f4160071040 0x7f4160071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816327095 0 0) 0x7f4160071950 con 0x7f416006dee0 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f414c0035b0 con 0x7f416006dee0 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669322573 0 0) 0x7f416015d060 con 0x7f4160160540 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4160071950 con 0x7f4160160540 2023-11-28T11:56:34.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1639227794 0 0) 0x7f414c0035b0 con 0x7f416006dee0 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41662f3700 1 -- 172.21.15.144:0/1793476523 >> v1:172.21.15.177:6789/0 conn(0x7f4160160540 legacy=0x7f4160160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41662f3700 1 -- 172.21.15.144:0/1793476523 >> v1:172.21.15.144:6789/0 conn(0x7f416006dee0 legacy=0x7f416006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41662f3700 1 -- 172.21.15.144:0/1793476523 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416015d620 con 0x7f4160071040 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41662f3700 1 --2- 172.21.15.144:0/1793476523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4160071040 0x7f4160071410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4150009840 tx=0x7f4150009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1def8de3bcdb1044 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c001670 con 0x7f4160071040 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4150004030 con 0x7f4160071040 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.323+0000 7f41652f1700 1 -- 172.21.15.144:0/1793476523 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41500043c0 con 0x7f4160071040 2023-11-28T11:56:34.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 -- 172.21.15.144:0/1793476523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4160071040 msgr2=0x7f4160071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 --2- 172.21.15.144:0/1793476523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4160071040 0x7f4160071410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4150009840 tx=0x7f4150009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:34.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 -- 172.21.15.144:0/1793476523 shutdown_connections 2023-11-28T11:56:34.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 --2- 172.21.15.144:0/1793476523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4160071040 0x7f4160071410 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 -- 172.21.15.144:0/1793476523 >> 172.21.15.144:0/1793476523 conn(0x7f41600d0000 msgr2=0x7f41600d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:34.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 -- 172.21.15.144:0/1793476523 shutdown_connections 2023-11-28T11:56:34.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.324+0000 7f4168557700 1 -- 172.21.15.144:0/1793476523 wait complete. 2023-11-28T11:56:34.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.325+0000 7f4168557700 1 Processor -- start 2023-11-28T11:56:34.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.325+0000 7f4168557700 1 -- start start 2023-11-28T11:56:34.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4168557700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416006dee0 0x7f4160156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4168557700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 0x7f41600b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4168557700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4160160540 0x7f4160158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4168557700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600c5020 con 0x7f4160071040 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4168557700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600c5300 con 0x7f416006dee0 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4168557700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41600c55e0 con 0x7f4160160540 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 0x7f41600b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 0x7f41600b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39534/0 (socket says 172.21.15.144:39534) 2023-11-28T11:56:34.333 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 -- 172.21.15.144:0/1139850413 learned_addr learned my addr 172.21.15.144:0/1139850413 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f41662f3700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416006dee0 0x7f4160156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4166af4700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4160160540 0x7f4160158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 -- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4160160540 msgr2=0x7f4160158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4160160540 0x7f4160158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 -- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416006dee0 msgr2=0x7f4160156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.326+0000 7f4165af2700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416006dee0 0x7f4160156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4165af2700 1 -- 172.21.15.144:0/1139850413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500094a0 con 0x7f4160071040 2023-11-28T11:56:34.334 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f41662f3700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416006dee0 0x7f4160156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4165af2700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 0x7f41600b2ec0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f415c00c6b0 tx=0x7f415c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c004c80 con 0x7f4160071040 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c004de0 con 0x7f4160071040 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416015ccd0 con 0x7f4160071040 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c0072f0 con 0x7f4160071040 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.327+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416015d900 con 0x7f4160071040 2023-11-28T11:56:34.335 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.328+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416006a270 con 0x7f4160071040 2023-11-28T11:56:34.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.328+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f415c01a020 con 0x7f4160071040 2023-11-28T11:56:34.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.329+0000 7f4156ffd700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f41440682f0 0x7f414406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:34.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.329+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f415c085650 con 0x7f4160071040 2023-11-28T11:56:34.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.329+0000 7f41662f3700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f41440682f0 0x7f414406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:34.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.329+0000 7f41662f3700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f41440682f0 0x7f414406a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41500090b0 tx=0x7f4150013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:34.336 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.333+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f415c0521b0 con 0x7f4160071040 2023-11-28T11:56:34.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.515+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41600720a0 con 0x7f4160071040 2023-11-28T11:56:34.516 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.516+0000 7f4156ffd700 1 -- 172.21.15.144:0/1139850413 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+14389 (secure 0 0 0) 0x7f415c0557d0 con 0x7f4160071040 2023-11-28T11:56:34.517 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:34.517 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":65,"fsid":"af726c13-c3f1-4279-9fca-12c6e4621b18","created":"2023-11-28T11:50:58.934818+0000","modified":"2023-11-28T11:56:31.093659+0000","last_up_change":"2023-11-28T11:51:09.423704+0000","last_in_change":"2023-11-28T11:51:01.426199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:51:10.205259+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:51:10.523478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04af8260-efb0-4064-89be-509ad55c21c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":103558},{"type":"v1","addr":"172.21.15.144:6817","nonce":103558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":103558},{"type":"v1","addr":"172.21.15.144:6819","nonce":103558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":103558},{"type":"v1","addr":"172.21.15.144:6824","nonce":103558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":103558},{"type":"v1","addr":"172.21.15.144:6821","nonce":103558}]},"public_addr":"172.21.15.144:6817/103558","cluster_addr":"172.21.15.144:6819/103558","heartbeat_back_addr":"172.21.15.144:6824/103558","heartbeat_front_addr":"172.21.15.144:6821/103558","state":["exists","up"]},{"osd":1,"uuid":"8fdb2107-bea4-4a2c-b202-a51d01193e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":103557},{"type":"v1","addr":"172.21.15.144:6809","nonce":103557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":103557},{"type":"v1","addr":"172.21.15.144:6811","nonce":103557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":103557},{"type":"v1","addr":"172.21.15.144:6815","nonce":103557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":103557},{"type":"v1","addr":"172.21.15.144:6813","nonce":103557}]},"public_addr":"172.21.15.144:6809/103557","cluster_addr":"172.21.15.144:6811/103557","heartbeat_back_addr":"172.21.15.144:6815/103557","heartbeat_front_addr":"172.21.15.144:6813/103557","state":["exists","up"]},{"osd":2,"uuid":"9ca99a48-48b5-41bc-bade-5d9fdb9bef65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":103559},{"type":"v1","addr":"172.21.15.144:6801","nonce":103559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":103559},{"type":"v1","addr":"172.21.15.144:6803","nonce":103559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":103559},{"type":"v1","addr":"172.21.15.144:6807","nonce":103559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":103559},{"type":"v1","addr":"172.21.15.144:6805","nonce":103559}]},"public_addr":"172.21.15.144:6801/103559","cluster_addr":"172.21.15.144:6803/103559","heartbeat_back_addr":"172.21.15.144:6807/103559","heartbeat_front_addr":"172.21.15.144:6805/103559","state":["exists","up"]},{"osd":3,"uuid":"c5f53b05-55f1-4c8f-aec4-40722b4132ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6823","nonce":103575},{"type":"v1","addr":"172.21.15.144:6825","nonce":103575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":103575},{"type":"v1","addr":"172.21.15.144:6827","nonce":103575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":103575},{"type":"v1","addr":"172.21.15.144:6831","nonce":103575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":103575},{"type":"v1","addr":"172.21.15.144:6829","nonce":103575}]},"public_addr":"172.21.15.144:6825/103575","cluster_addr":"172.21.15.144:6827/103575","heartbeat_back_addr":"172.21.15.144:6831/103575","heartbeat_front_addr":"172.21.15.144:6829/103575","state":["exists","up"]},{"osd":4,"uuid":"4f434f89-a39d-454a-9e0b-2b9fcbe3cce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":103772},{"type":"v1","addr":"172.21.15.177:6809","nonce":103772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":103772},{"type":"v1","addr":"172.21.15.177:6811","nonce":103772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":103772},{"type":"v1","addr":"172.21.15.177:6815","nonce":103772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":103772},{"type":"v1","addr":"172.21.15.177:6813","nonce":103772}]},"public_addr":"172.21.15.177:6809/103772","cluster_addr":"172.21.15.177:6811/103772","heartbeat_back_addr":"172.21.15.177:6815/103772","heartbeat_front_addr":"172.21.15.177:6813/103772","state":["exists","up"]},{"osd":5,"uuid":"3ba5d415-e46a-4d3e-b94f-86521b067ea1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6817","nonce":103774},{"type":"v1","addr":"172.21.15.177:6819","nonce":103774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6821","nonce":103774},{"type":"v1","addr":"172.21.15.177:6823","nonce":103774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6829","nonce":103774},{"type":"v1","addr":"172.21.15.177:6831","nonce":103774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6825","nonce":103774},{"type":"v1","addr":"172.21.15.177:6827","nonce":103774}]},"public_addr":"172.21.15.177:6819/103774","cluster_addr":"172.21.15.177:6823/103774","heartbeat_back_addr":"172.21.15.177:6831/103774","heartbeat_front_addr":"172.21.15.177:6827/103774","state":["exists","up"]},{"osd":6,"uuid":"807086dc-2ec0-487a-af31-01d27a3db051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":103771},{"type":"v1","addr":"172.21.15.177:6801","nonce":103771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":103771},{"type":"v1","addr":"172.21.15.177:6803","nonce":103771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":103771},{"type":"v1","addr":"172.21.15.177:6807","nonce":103771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":103771},{"type":"v1","addr":"172.21.15.177:6805","nonce":103771}]},"public_addr":"172.21.15.177:6801/103771","cluster_addr":"172.21.15.177:6803/103771","heartbeat_back_addr":"172.21.15.177:6807/103771","heartbeat_front_addr":"172.21.15.177:6805/103771","state":["exists","up"]},{"osd":7,"uuid":"12c21e8c-9c83-4d7f-8baf-f0154e97b9ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":103773},{"type":"v1","addr":"172.21.15.177:6818","nonce":103773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":103773},{"type":"v1","addr":"172.21.15.177:6822","nonce":103773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":103773},{"type":"v1","addr":"172.21.15.177:6830","nonce":103773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":103773},{"type":"v1","addr":"172.21.15.177:6826","nonce":103773}]},"public_addr":"172.21.15.177:6818/103773","cluster_addr":"172.21.15.177:6822/103773","heartbeat_back_addr":"172.21.15.177:6830/103773","heartbeat_front_addr":"172.21.15.177:6826/103773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.655818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.638411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.409820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:08.421794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.599092+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.620992+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.362079+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:51:06.676913+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6837/3077123372":"2023-11-29T11:56:27.283398+0000","172.21.15.144:6835/3077123372":"2023-11-29T11:56:27.283398+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:56:34.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f41440682f0 msgr2=0x7f414406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f41440682f0 0x7f414406a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f41500090b0 tx=0x7f4150013040 comp rx=0 tx=0).stop 2023-11-28T11:56:34.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 msgr2=0x7f41600b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:34.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 0x7f41600b2ec0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f415c00c6b0 tx=0x7f415c00ca70 comp rx=0 tx=0).stop 2023-11-28T11:56:34.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 shutdown_connections 2023-11-28T11:56:34.519 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f41440682f0 0x7f414406a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4160160540 0x7f4160158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f416006dee0 0x7f4160156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 --2- 172.21.15.144:0/1139850413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4160071040 0x7f41600b2ec0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 >> 172.21.15.144:0/1139850413 conn(0x7f41600d0000 msgr2=0x7f416015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 shutdown_connections 2023-11-28T11:56:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:34.518+0000 7f4168557700 1 -- 172.21.15.144:0/1139850413 wait complete. 2023-11-28T11:56:35.532 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T11:56:35.532 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:35.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.669+0000 7ff78793d700 1 Processor -- start 2023-11-28T11:56:35.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.669+0000 7ff78793d700 1 -- start start 2023-11-28T11:56:35.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff78793d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff78015c6e0 0x7ff7800cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff78793d700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7800d0120 0x7ff7800cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff78793d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800cc820 con 0x7ff7800cc3a0 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff78793d700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff780066600 con 0x7ff7800d0120 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff78793d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78006b3b0 con 0x7ff78015c6e0 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7800d0120 0x7ff7800cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7800d0120 0x7ff7800cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39540/0 (socket says 172.21.15.144:39540) 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 -- 172.21.15.144:0/2536630187 learned_addr learned my addr 172.21.15.144:0/2536630187 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff7856d9700 1 --2- 172.21.15.144:0/2536630187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff78015c6e0 0x7ff7800cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 -- 172.21.15.144:0/2536630187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff78015c6e0 msgr2=0x7ff7800cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 --2- 172.21.15.144:0/2536630187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff78015c6e0 0x7ff7800cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 -- 172.21.15.144:0/2536630187 >> v1:172.21.15.177:6789/0 conn(0x7ff7800cc3a0 legacy=0x7ff780065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.670+0000 7ff784ed8700 1 -- 172.21.15.144:0/2536630187 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800ced60 con 0x7ff7800d0120 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff784ed8700 1 --2- 172.21.15.144:0/2536630187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7800d0120 0x7ff7800cbe60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00bb90 tx=0x7ff77c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf16a3cf128bca8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff777fff700 1 -- 172.21.15.144:0/2536630187 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c00dd10 con 0x7ff7800d0120 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff777fff700 1 -- 172.21.15.144:0/2536630187 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c00de70 con 0x7ff7800d0120 2023-11-28T11:56:35.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff777fff700 1 -- 172.21.15.144:0/2536630187 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c004230 con 0x7ff7800d0120 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 -- 172.21.15.144:0/2536630187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7800d0120 msgr2=0x7ff7800cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 --2- 172.21.15.144:0/2536630187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7800d0120 0x7ff7800cbe60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00bb90 tx=0x7ff77c00bea0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 -- 172.21.15.144:0/2536630187 shutdown_connections 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 --2- 172.21.15.144:0/2536630187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff78015c6e0 0x7ff7800cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 --2- 172.21.15.144:0/2536630187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7800d0120 0x7ff7800cbe60 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 -- 172.21.15.144:0/2536630187 >> 172.21.15.144:0/2536630187 conn(0x7ff7800b3080 msgr2=0x7ff7800cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:35.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 -- 172.21.15.144:0/2536630187 shutdown_connections 2023-11-28T11:56:35.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.671+0000 7ff78793d700 1 -- 172.21.15.144:0/2536630187 wait complete. 2023-11-28T11:56:35.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 Processor -- start 2023-11-28T11:56:35.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 -- start start 2023-11-28T11:56:35.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7800cc3a0 0x7ff7800c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800d0120 0x7ff780098a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff78015c6e0 0x7ff7800c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c4460 con 0x7ff78015c6e0 2023-11-28T11:56:35.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c3850 con 0x7ff7800d0120 2023-11-28T11:56:35.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.673+0000 7ff78793d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c3b30 con 0x7ff7800cc3a0 2023-11-28T11:56:35.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff78015c6e0 0x7ff7800c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff78015c6e0 0x7ff7800c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39548/0 (socket says 172.21.15.144:39548) 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 -- 172.21.15.144:0/1536226271 learned_addr learned my addr 172.21.15.144:0/1536226271 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff784ed8700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800d0120 0x7ff780098a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff7856d9700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7800cc3a0 0x7ff7800c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 -- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7800cc3a0 msgr2=0x7ff7800c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7800cc3a0 0x7ff7800c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 -- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800d0120 msgr2=0x7ff780098a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.681 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7800d0120 0x7ff780098a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 -- 172.21.15.144:0/1536226271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c00b7f0 con 0x7ff78015c6e0 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff7856d9700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7800cc3a0 0x7ff7800c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff785eda700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff78015c6e0 0x7ff7800c3000 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff77800b970 tx=0x7ff77800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800c930 con 0x7ff78015c6e0 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78006a5f0 con 0x7ff78015c6e0 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77800ca90 con 0x7ff78015c6e0 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77800ce40 con 0x7ff78015c6e0 2023-11-28T11:56:35.682 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.674+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78006a310 con 0x7ff78015c6e0 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.675+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff78006a8d0 con 0x7ff78015c6e0 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff77801b030 con 0x7ff78015c6e0 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff775ffb700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7640682f0 0x7ff76406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff778085450 con 0x7ff78015c6e0 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff7856d9700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7640682f0 0x7ff76406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff775ffb700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7ff76406e760 0x7ff764070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.683 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 --> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff764071240 con 0x7ff76406e760 2023-11-28T11:56:35.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff778087b00 con 0x7ff78015c6e0 2023-11-28T11:56:35.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff784ed8700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7ff76406e760 0x7ff764070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.676+0000 7ff7856d9700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7640682f0 0x7ff76406a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff7700099f0 tx=0x7ff770008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.677+0000 7ff784ed8700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7ff76406e760 0x7ff764070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.684 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.677+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== osd.0 v2:172.21.15.144:6816/103558 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff764071240 con 0x7ff76406e760 2023-11-28T11:56:35.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.691+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 --> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff7800c3ca0 con 0x7ff76406e760 2023-11-28T11:56:35.692 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.691+0000 7ff775ffb700 1 -- 172.21.15.144:0/1536226271 <== osd.0 v2:172.21.15.144:6816/103558 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff7800c3ca0 con 0x7ff76406e760 2023-11-28T11:56:35.692 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:35.693 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:35.693 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:35.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7ff76406e760 msgr2=0x7ff764070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6816/103558,v1:172.21.15.144:6817/103558] conn(0x7ff76406e760 0x7ff764070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.694 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7640682f0 msgr2=0x7ff76406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7ff7640682f0 0x7ff76406a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff7700099f0 tx=0x7ff770008040 comp rx=0 tx=0).stop 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff78015c6e0 msgr2=0x7ff7800c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 --2- 172.21.15.144:0/1536226271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff78015c6e0 0x7ff7800c3000 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff77800b970 tx=0x7ff77800bc80 comp rx=0 tx=0).stop 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff785eda700 1 -- 172.21.15.144:0/1536226271 reap_dead start 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 shutdown_connections 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 >> 172.21.15.144:0/1536226271 conn(0x7ff7800b3080 msgr2=0x7ff780159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 shutdown_connections 2023-11-28T11:56:35.695 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.692+0000 7ff78793d700 1 -- 172.21.15.144:0/1536226271 wait complete. 2023-11-28T11:56:35.704 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T11:56:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.825+0000 7f8af5f23700 1 Processor -- start 2023-11-28T11:56:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.825+0000 7f8af5f23700 1 -- start start 2023-11-28T11:56:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8af5f23700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8af0071040 0x7f8af0071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8af5f23700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af0071950 con 0x7f8af0160540 2023-11-28T11:56:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8af5f23700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af015d060 con 0x7f8af006dee0 2023-11-28T11:56:35.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8af5f23700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af015d340 con 0x7f8af0071040 2023-11-28T11:56:35.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aef7fe700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8af0071040 0x7f8af0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aeffff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8af0160540 0x7f8af0160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56266/0 (socket says 172.21.15.144:56266) 2023-11-28T11:56:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aeffff700 1 -- 172.21.15.144:0/4175843509 learned_addr learned my addr 172.21.15.144:0/4175843509 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aef7fe700 1 -- 172.21.15.144:0/4175843509 >> v1:172.21.15.177:6789/0 conn(0x7f8af006dee0 legacy=0x7f8af006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aef7fe700 1 -- 172.21.15.144:0/4175843509 >> v1:172.21.15.144:6789/0 conn(0x7f8af0160540 legacy=0x7f8af0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aef7fe700 1 -- 172.21.15.144:0/4175843509 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af015d620 con 0x7f8af0071040 2023-11-28T11:56:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.826+0000 7f8aee7fc700 1 -- 172.21.15.144:0/4175843509 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647074408 0 0) 0x7f8af0071950 con 0x7f8af0160540 2023-11-28T11:56:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8aef7fe700 1 --2- 172.21.15.144:0/4175843509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af0071040 0x7f8af0071410 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009840 tx=0x7f8ae4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685b1a4548c89a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8aee7fc700 1 -- 172.21.15.144:0/4175843509 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4004230 con 0x7f8af0071040 2023-11-28T11:56:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8aee7fc700 1 -- 172.21.15.144:0/4175843509 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4004390 con 0x7f8af0071040 2023-11-28T11:56:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8aee7fc700 1 -- 172.21.15.144:0/4175843509 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40036a0 con 0x7f8af0071040 2023-11-28T11:56:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 -- 172.21.15.144:0/4175843509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af0071040 msgr2=0x7f8af0071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 --2- 172.21.15.144:0/4175843509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af0071040 0x7f8af0071410 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009840 tx=0x7f8ae4009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:35.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 -- 172.21.15.144:0/4175843509 shutdown_connections 2023-11-28T11:56:35.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 --2- 172.21.15.144:0/4175843509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af0071040 0x7f8af0071410 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 -- 172.21.15.144:0/4175843509 >> 172.21.15.144:0/4175843509 conn(0x7f8af00d0000 msgr2=0x7f8af00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:35.832 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 -- 172.21.15.144:0/4175843509 shutdown_connections 2023-11-28T11:56:35.833 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.827+0000 7f8af5f23700 1 -- 172.21.15.144:0/4175843509 wait complete. 2023-11-28T11:56:35.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.828+0000 7f8af5f23700 1 Processor -- start 2023-11-28T11:56:35.837 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 -- start start 2023-11-28T11:56:35.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 0x7f8af00c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8af0071040 0x7f8af01f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8af0160540 0x7f8af00c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.838 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af01f25f0 con 0x7f8af006dee0 2023-11-28T11:56:35.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00c4a80 con 0x7f8af0071040 2023-11-28T11:56:35.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8af5f23700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00c68d0 con 0x7f8af0160540 2023-11-28T11:56:35.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.829+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 0x7f8af00c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.839 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 0x7f8af00c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39558/0 (socket says 172.21.15.144:39558) 2023-11-28T11:56:35.840 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 -- 172.21.15.144:0/2167112853 learned_addr learned my addr 172.21.15.144:0/2167112853 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:35.841 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aeeffd700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8af0071040 0x7f8af01f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aeffff700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8af0160540 0x7f8af00c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 -- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8af0160540 msgr2=0x7f8af00c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8af0160540 0x7f8af00c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.842 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 -- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8af0071040 msgr2=0x7f8af01f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:35.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8af0071040 0x7f8af01f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:35.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 -- 172.21.15.144:0/2167112853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40094a0 con 0x7f8af006dee0 2023-11-28T11:56:35.843 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aeffff700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8af0160540 0x7f8af00c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:35.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8aef7fe700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 0x7f8af00c56e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009790 tx=0x7f8ae40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4003e90 con 0x7f8af006dee0 2023-11-28T11:56:35.844 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4011630 con 0x7f8af006dee0 2023-11-28T11:56:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af015ccd0 con 0x7f8af006dee0 2023-11-28T11:56:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40119c0 con 0x7f8af006dee0 2023-11-28T11:56:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.830+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af015d900 con 0x7f8af006dee0 2023-11-28T11:56:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.831+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af006a270 con 0x7f8af006dee0 2023-11-28T11:56:35.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.831+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ae4017020 con 0x7f8af006dee0 2023-11-28T11:56:35.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.832+0000 7f8ad7fff700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8adc0682f0 0x7f8adc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:35.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.832+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8ae4087020 con 0x7f8af006dee0 2023-11-28T11:56:35.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.832+0000 7f8aeeffd700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8adc0682f0 0x7f8adc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:35.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.832+0000 7f8aeeffd700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8adc0682f0 0x7f8adc06a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8af00c5040 tx=0x7f8ae00090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:35.847 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:35.836+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ae4053b80 con 0x7f8af006dee0 2023-11-28T11:56:36.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.021+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8af01f27c0 con 0x7f8adc0682f0 2023-11-28T11:56:36.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.022+0000 7f8ad7fff700 1 -- 172.21.15.144:0/2167112853 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8af01f27c0 con 0x7f8adc0682f0 2023-11-28T11:56:36.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8adc0682f0 msgr2=0x7f8adc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8adc0682f0 0x7f8adc06a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8af00c5040 tx=0x7f8ae00090e0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 msgr2=0x7f8af00c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 0x7f8af00c56e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009790 tx=0x7f8ae40047c0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 shutdown_connections 2023-11-28T11:56:36.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8af006dee0 0x7f8af00c56e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8adc0682f0 0x7f8adc06a7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8af0160540 0x7f8af00c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 --2- 172.21.15.144:0/2167112853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8af0071040 0x7f8af01f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 >> 172.21.15.144:0/2167112853 conn(0x7f8af00d0000 msgr2=0x7f8af00af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 shutdown_connections 2023-11-28T11:56:36.026 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.024+0000 7f8af5f23700 1 -- 172.21.15.144:0/2167112853 wait complete. 2023-11-28T11:56:36.027 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T11:56:36.037 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T11:56:36.037 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:36.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.160+0000 7f5e2ee1a700 1 Processor -- start 2023-11-28T11:56:36.162 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.160+0000 7f5e2ee1a700 1 -- start start 2023-11-28T11:56:36.169 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e2ee1a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 0x7f5e280b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e2ee1a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5e280d0140 0x7f5e280d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.170 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e2ee1a700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2815af70 con 0x7f5e280b6ae0 2023-11-28T11:56:36.171 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e2ee1a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e280d0b00 con 0x7f5e280d0140 2023-11-28T11:56:36.171 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e2ee1a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280cfec0 con 0x7f5e2815aba0 2023-11-28T11:56:36.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e27fff700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5e280b6ae0 0x7f5e280cf4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56278/0 (socket says 172.21.15.144:56278) 2023-11-28T11:56:36.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.161+0000 7f5e27fff700 1 -- 172.21.15.144:0/2398434671 learned_addr learned my addr 172.21.15.144:0/2398434671 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:36.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588677939 0 0) 0x7f5e2815af70 con 0x7f5e280b6ae0 2023-11-28T11:56:36.172 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/2398434671 >> v2:172.21.15.177:3300/0 conn(0x7f5e280d0140 0x7f5e280d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2cbb6700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 0x7f5e280b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e180035b0 con 0x7f5e280b6ae0 2023-11-28T11:56:36.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873166010 0 0) 0x7f5e180035b0 con 0x7f5e280b6ae0 2023-11-28T11:56:36.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 msgr2=0x7f5e280b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 0x7f5e280b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 >> v2:172.21.15.177:3300/0 conn(0x7f5e280d0140 msgr2=0x7f5e280d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 --2- 172.21.15.144:0/2398434671 >> v2:172.21.15.177:3300/0 conn(0x7f5e280d0140 0x7f5e280d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280cc870 con 0x7f5e280b6ae0 2023-11-28T11:56:36.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f5e14001670 con 0x7f5e280b6ae0 2023-11-28T11:56:36.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 >> v1:172.21.15.144:6789/0 conn(0x7f5e280b6ae0 legacy=0x7f5e280cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18003a00 0x7f5e18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 0x7f5e18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e18004d90 0x7f5e18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280d0b00 con 0x7f5e18004d90 2023-11-28T11:56:36.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280cfec0 con 0x7f5e18003a00 2023-11-28T11:56:36.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1800ba10 con 0x7f5e2815aba0 2023-11-28T11:56:36.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e18004d90 0x7f5e18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2cbb6700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18003a00 0x7f5e18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e27fff700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 0x7f5e18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 msgr2=0x7f5e18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815aba0 0x7f5e18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18003a00 msgr2=0x7f5e18003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e18003a00 0x7f5e18003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.162+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/2398434671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180035b0 con 0x7f5e18004d90 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e18004d90 0x7f5e18005200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009fc0 tx=0x7f5e18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1181c40cf6d5a65e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000ebd0 con 0x7f5e18004d90 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2000f040 con 0x7f5e18004d90 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e277fe700 1 -- 172.21.15.144:0/2398434671 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20014480 con 0x7f5e18004d90 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/2398434671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e18004d90 msgr2=0x7f5e18005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2ee1a700 1 --2- 172.21.15.144:0/2398434671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e18004d90 0x7f5e18005200 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009fc0 tx=0x7f5e18005880 comp rx=0 tx=0).stop 2023-11-28T11:56:36.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/2398434671 reap_dead start 2023-11-28T11:56:36.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/2398434671 shutdown_connections 2023-11-28T11:56:36.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/2398434671 >> 172.21.15.144:0/2398434671 conn(0x7f5e2806df70 msgr2=0x7f5e2806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/2398434671 shutdown_connections 2023-11-28T11:56:36.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.163+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/2398434671 wait complete. 2023-11-28T11:56:36.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.164+0000 7f5e2ee1a700 1 Processor -- start 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.164+0000 7f5e2ee1a700 1 -- start start 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2ee1a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2814d3f0 0x7f5e280b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2ee1a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815af70 0x7f5e280b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2ee1a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2814d7c0 0x7f5e280ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2ee1a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280afe50 con 0x7f5e2814d7c0 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2ee1a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2806c050 con 0x7f5e2814d3f0 2023-11-28T11:56:36.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2ee1a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280cf950 con 0x7f5e2815af70 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.165+0000 7f5e2d3b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2814d7c0 0x7f5e280ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2814d7c0 0x7f5e280ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39580/0 (socket says 172.21.15.144:39580) 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/1645442795 learned_addr learned my addr 172.21.15.144:0/1645442795 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2cbb6700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2814d3f0 0x7f5e280b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e27fff700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815af70 0x7f5e280b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/1645442795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815af70 msgr2=0x7f5e280b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.181 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e2815af70 0x7f5e280b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/1645442795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2814d3f0 msgr2=0x7f5e280b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e2814d3f0 0x7f5e280b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/1645442795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280ccaa0 con 0x7f5e2814d7c0 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2d3b7700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2814d7c0 0x7f5e280ce7b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5e20016000 tx=0x7f5e20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20007000 con 0x7f5e2814d7c0 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e20004450 con 0x7f5e2814d7c0 2023-11-28T11:56:36.182 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e280cfbf0 con 0x7f5e2814d7c0 2023-11-28T11:56:36.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20004800 con 0x7f5e2814d7c0 2023-11-28T11:56:36.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.166+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e280ccd80 con 0x7f5e2814d7c0 2023-11-28T11:56:36.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.167+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e28068690 con 0x7f5e2814d7c0 2023-11-28T11:56:36.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e20014480 con 0x7f5e2814d7c0 2023-11-28T11:56:36.183 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e257fa700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5e20086ab0 con 0x7f5e2814d7c0 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e2cbb6700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e257fa700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f5e0c06e760 0x7f5e0c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e0c071240 con 0x7f5e0c06e760 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2008b2a0 con 0x7f5e2814d7c0 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e27fff700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f5e0c06e760 0x7f5e0c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.184 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e2cbb6700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009be0 tx=0x7f5e1c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.168+0000 7f5e27fff700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f5e0c06e760 0x7f5e0c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.185 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.169+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== osd.1 v2:172.21.15.144:6808/103557 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e0c071240 con 0x7f5e0c06e760 2023-11-28T11:56:36.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.183+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 --> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5e2806c1c0 con 0x7f5e0c06e760 2023-11-28T11:56:36.186 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.183+0000 7f5e257fa700 1 -- 172.21.15.144:0/1645442795 <== osd.1 v2:172.21.15.144:6808/103557 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5e2806c1c0 con 0x7f5e0c06e760 2023-11-28T11:56:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:36.186 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:36.187 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:36.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.183+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f5e0c06e760 msgr2=0x7f5e0c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.183+0000 7f5e2ee1a700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6808/103557,v1:172.21.15.144:6809/103557] conn(0x7f5e0c06e760 0x7f5e0c070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e0c0682f0 msgr2=0x7f5e0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009be0 tx=0x7f5e1c009210 comp rx=0 tx=0).stop 2023-11-28T11:56:36.188 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2814d7c0 msgr2=0x7f5e280ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 --2- 172.21.15.144:0/1645442795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e2814d7c0 0x7f5e280ce7b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5e20016000 tx=0x7f5e20009750 comp rx=0 tx=0).stop 2023-11-28T11:56:36.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2d3b7700 1 -- 172.21.15.144:0/1645442795 reap_dead start 2023-11-28T11:56:36.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 shutdown_connections 2023-11-28T11:56:36.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 >> 172.21.15.144:0/1645442795 conn(0x7f5e2806df70 msgr2=0x7f5e280b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 shutdown_connections 2023-11-28T11:56:36.189 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.184+0000 7f5e2ee1a700 1 -- 172.21.15.144:0/1645442795 wait complete. 2023-11-28T11:56:36.196 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T11:56:36.316 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.315+0000 7f4cf4cf2700 1 Processor -- start 2023-11-28T11:56:36.316 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.315+0000 7f4cf4cf2700 1 -- start start 2023-11-28T11:56:36.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.316+0000 7f4cf4cf2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4cf006dee0 0x7f4cf006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.316+0000 7f4cf4cf2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf015d060 con 0x7f4cf0071040 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.316+0000 7f4cf4cf2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf015d340 con 0x7f4cf0160540 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.316+0000 7f4cf4cf2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf015d620 con 0x7f4cf006dee0 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cee59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4cf0071040 0x7f4cf0071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56300/0 (socket says 172.21.15.144:56300) 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cee59c700 1 -- 172.21.15.144:0/1744235662 learned_addr learned my addr 172.21.15.144:0/1744235662 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cedd9b700 1 --2- 172.21.15.144:0/1744235662 >> v2:172.21.15.177:3300/0 conn(0x7f4cf006dee0 0x7f4cf006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4ced59a700 1 -- 172.21.15.144:0/1744235662 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114907748 0 0) 0x7f4cf015d060 con 0x7f4cf0071040 2023-11-28T11:56:36.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4ced59a700 1 -- 172.21.15.144:0/1744235662 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd80035b0 con 0x7f4cf0071040 2023-11-28T11:56:36.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cedd9b700 1 -- 172.21.15.144:0/1744235662 >> v1:172.21.15.177:6789/0 conn(0x7f4cf0160540 legacy=0x7f4cf0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4ced59a700 1 -- 172.21.15.144:0/1744235662 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881979847 0 0) 0x7f4cd80035b0 con 0x7f4cf0071040 2023-11-28T11:56:36.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cedd9b700 1 -- 172.21.15.144:0/1744235662 >> v1:172.21.15.144:6789/0 conn(0x7f4cf0071040 legacy=0x7f4cf0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cedd9b700 1 -- 172.21.15.144:0/1744235662 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf015d900 con 0x7f4cf006dee0 2023-11-28T11:56:36.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4cedd9b700 1 --2- 172.21.15.144:0/1744235662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf006dee0 0x7f4cf006e330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009840 tx=0x7f4ce0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcc6e76259ff684 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4ced59a700 1 -- 172.21.15.144:0/1744235662 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4001670 con 0x7f4cf006dee0 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4ced59a700 1 -- 172.21.15.144:0/1744235662 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0004030 con 0x7f4cf006dee0 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.317+0000 7f4ced59a700 1 -- 172.21.15.144:0/1744235662 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00043c0 con 0x7f4cf006dee0 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/1744235662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf006dee0 msgr2=0x7f4cf006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/1744235662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf006dee0 0x7f4cf006e330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009840 tx=0x7f4ce0009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/1744235662 shutdown_connections 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/1744235662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf006dee0 0x7f4cf006e330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/1744235662 >> 172.21.15.144:0/1744235662 conn(0x7f4cf00d0000 msgr2=0x7f4cf00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/1744235662 shutdown_connections 2023-11-28T11:56:36.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.318+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/1744235662 wait complete. 2023-11-28T11:56:36.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.319+0000 7f4cf4cf2700 1 Processor -- start 2023-11-28T11:56:36.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.319+0000 7f4cf4cf2700 1 -- start start 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cf4cf2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 0x7f4cf0144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cf4cf2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf0071040 0x7f4cf0141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cf4cf2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cf0160540 0x7f4cf01f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cf4cf2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00b4c60 con 0x7f4cf006dee0 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cf4cf2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00c6210 con 0x7f4cf0071040 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cf4cf2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0144f90 con 0x7f4cf0160540 2023-11-28T11:56:36.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cee59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 0x7f4cf0144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cee59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 0x7f4cf0144a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39594/0 (socket says 172.21.15.144:39594) 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cee59c700 1 -- 172.21.15.144:0/2320012101 learned_addr learned my addr 172.21.15.144:0/2320012101 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cedd9b700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf0071040 0x7f4cf0141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4cee59c700 1 -- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cf0160540 msgr2=0x7f4cf01f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.320+0000 7f4ceed9d700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cf0160540 0x7f4cf01f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cee59c700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cf0160540 0x7f4cf01f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cee59c700 1 -- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf0071040 msgr2=0x7f4cf0141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cee59c700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf0071040 0x7f4cf0141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cee59c700 1 -- 172.21.15.144:0/2320012101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00094a0 con 0x7f4cf006dee0 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cedd9b700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf0071040 0x7f4cf0141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4ceed9d700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cf0160540 0x7f4cf01f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cee59c700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 0x7f4cf0144a50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c550 tx=0x7f4ce400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4004c80 con 0x7f4cf006dee0 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4004de0 con 0x7f4cf006dee0 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf015ccd0 con 0x7f4cf006dee0 2023-11-28T11:56:36.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4007310 con 0x7f4cf006dee0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.321+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf006e870 con 0x7f4cf006dee0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.322+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf006a270 con 0x7f4cf006dee0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.322+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ce401a020 con 0x7f4cf006dee0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.323+0000 7f4cdeffd700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4cd00682f0 0x7f4cd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.323+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ce4085650 con 0x7f4cf006dee0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.323+0000 7f4cedd9b700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4cd00682f0 0x7f4cd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.323+0000 7f4cedd9b700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4cd00682f0 0x7f4cd006a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00aeea0 tx=0x7f4ce0013040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.327+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4ce4052260 con 0x7f4cf006dee0 2023-11-28T11:56:36.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.509+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cf00720a0 con 0x7f4cd00682f0 2023-11-28T11:56:36.510 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.509+0000 7f4cdeffd700 1 -- 172.21.15.144:0/2320012101 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4cf00720a0 con 0x7f4cd00682f0 2023-11-28T11:56:36.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4cd00682f0 msgr2=0x7f4cd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4cd00682f0 0x7f4cd006a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00aeea0 tx=0x7f4ce0013040 comp rx=0 tx=0).stop 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 msgr2=0x7f4cf0144a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 0x7f4cf0144a50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c550 tx=0x7f4ce400c910 comp rx=0 tx=0).stop 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 shutdown_connections 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf006dee0 0x7f4cf0144a50 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f4cd00682f0 0x7f4cd006a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4cf0160540 0x7f4cf01f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 --2- 172.21.15.144:0/2320012101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cf0071040 0x7f4cf0141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 >> 172.21.15.144:0/2320012101 conn(0x7f4cf00d0000 msgr2=0x7f4cf00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 shutdown_connections 2023-11-28T11:56:36.515 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.511+0000 7f4cf4cf2700 1 -- 172.21.15.144:0/2320012101 wait complete. 2023-11-28T11:56:36.515 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T11:56:36.524 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T11:56:36.524 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:36.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.649+0000 7f0c84bc2700 1 Processor -- start 2023-11-28T11:56:36.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.649+0000 7f0c84bc2700 1 -- start start 2023-11-28T11:56:36.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c84bc2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c800af430 0x7f0c80070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c84bc2700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800713c0 con 0x7f0c800af060 2023-11-28T11:56:36.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c84bc2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800716a0 con 0x7f0c8006a980 2023-11-28T11:56:36.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c84bc2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c80071980 con 0x7f0c800af430 2023-11-28T11:56:36.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7e59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0c800af060 0x7f0c8006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56314/0 (socket says 172.21.15.144:56314) 2023-11-28T11:56:36.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7e59c700 1 -- 172.21.15.144:0/3858396139 learned_addr learned my addr 172.21.15.144:0/3858396139 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:36.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/3858396139 >> v2:172.21.15.177:3300/0 conn(0x7f0c800af430 0x7f0c80070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895890628 0 0) 0x7f0c800713c0 con 0x7f0c800af060 2023-11-28T11:56:36.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c680035b0 con 0x7f0c800af060 2023-11-28T11:56:36.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822716118 0 0) 0x7f0c680035b0 con 0x7f0c800af060 2023-11-28T11:56:36.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 >> v2:172.21.15.177:3300/0 conn(0x7f0c800af430 msgr2=0x7f0c80070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 --2- 172.21.15.144:0/3858396139 >> v2:172.21.15.177:3300/0 conn(0x7f0c800af430 0x7f0c80070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 >> v1:172.21.15.177:6789/0 conn(0x7f0c8006a980 legacy=0x7f0c800ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80071c60 con 0x7f0c800af060 2023-11-28T11:56:36.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f0c70001670 con 0x7f0c800af060 2023-11-28T11:56:36.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 >> v1:172.21.15.144:6789/0 conn(0x7f0c800af060 legacy=0x7f0c8006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68003a00 0x7f0c68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c68004850 0x7f0c68004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c68005200 0x7f0c6800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80071980 con 0x7f0c68005200 2023-11-28T11:56:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6800c080 con 0x7f0c68004850 2023-11-28T11:56:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.650+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6800c240 con 0x7f0c68003a00 2023-11-28T11:56:36.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c68005200 0x7f0c6800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7dd9b700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c68004850 0x7f0c68004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7e59c700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68003a00 0x7f0c68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 -- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68003a00 msgr2=0x7f0c68003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68003a00 0x7f0c68003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 -- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c68004850 msgr2=0x7f0c68004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c68004850 0x7f0c68004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 -- 172.21.15.144:0/3858396139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c680035b0 con 0x7f0c68005200 2023-11-28T11:56:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 -- 172.21.15.144:0/3858396139 reap_dead start 2023-11-28T11:56:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c68005200 0x7f0c6800ba10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009fc0 tx=0x7f0c68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a7f396d2367fd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78003a20 con 0x7f0c68005200 2023-11-28T11:56:36.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7800f040 con 0x7f0c68005200 2023-11-28T11:56:36.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.651+0000 7f0c7d59a700 1 -- 172.21.15.144:0/3858396139 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78003da0 con 0x7f0c68005200 2023-11-28T11:56:36.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 -- 172.21.15.144:0/3858396139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c68005200 msgr2=0x7f0c6800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c68005200 0x7f0c6800ba10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009fc0 tx=0x7f0c68005880 comp rx=0 tx=0).stop 2023-11-28T11:56:36.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 -- 172.21.15.144:0/3858396139 shutdown_connections 2023-11-28T11:56:36.665 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 --2- 172.21.15.144:0/3858396139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c68005200 0x7f0c6800ba10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 -- 172.21.15.144:0/3858396139 >> 172.21.15.144:0/3858396139 conn(0x7f0c8006b380 msgr2=0x7f0c8006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 -- 172.21.15.144:0/3858396139 shutdown_connections 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.652+0000 7f0c84bc2700 1 -- 172.21.15.144:0/3858396139 wait complete. 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.653+0000 7f0c84bc2700 1 Processor -- start 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.653+0000 7f0c84bc2700 1 -- start start 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c84bc2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68005200 0x7f0c800b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c84bc2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c800acb90 0x7f0c8014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c84bc2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c800ad000 0x7f0c8014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c84bc2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80156720 con 0x7f0c800acb90 2023-11-28T11:56:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c84bc2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b1e10 con 0x7f0c800ad000 2023-11-28T11:56:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c84bc2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800af8d0 con 0x7f0c68005200 2023-11-28T11:56:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c800acb90 0x7f0c8014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c800acb90 0x7f0c8014cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39616/0 (socket says 172.21.15.144:39616) 2023-11-28T11:56:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 -- 172.21.15.144:0/1070201153 learned_addr learned my addr 172.21.15.144:0/1070201153 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c800ad000 0x7f0c8014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7e59c700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68005200 0x7f0c800b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 -- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68005200 msgr2=0x7f0c800b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68005200 0x7f0c800b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 -- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c800ad000 msgr2=0x7f0c8014d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c800ad000 0x7f0c8014d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 -- 172.21.15.144:0/1070201153 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80070d50 con 0x7f0c800acb90 2023-11-28T11:56:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c800ad000 0x7f0c8014d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:36.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7e59c700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c68005200 0x7f0c800b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:36.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c7dd9b700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c800acb90 0x7f0c8014cd30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400f610 tx=0x7f0c7400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.654+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c740052e0 con 0x7f0c800acb90 2023-11-28T11:56:36.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.655+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74005440 con 0x7f0c800acb90 2023-11-28T11:56:36.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.655+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c80071f40 con 0x7f0c800acb90 2023-11-28T11:56:36.669 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.655+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c740038a0 con 0x7f0c800acb90 2023-11-28T11:56:36.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.655+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c80070ee0 con 0x7f0c800acb90 2023-11-28T11:56:36.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.655+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c800713c0 con 0x7f0c800acb90 2023-11-28T11:56:36.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.656+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c74003a00 con 0x7f0c800acb90 2023-11-28T11:56:36.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.656+0000 7f0c6effd700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.656+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c74086230 con 0x7f0c800acb90 2023-11-28T11:56:36.670 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.656+0000 7f0c7e59c700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.657+0000 7f0c6effd700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f0c6006e760 0x7f0c60070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.657+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 --> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c60071240 con 0x7f0c6006e760 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.657+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c74086670 con 0x7f0c800acb90 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.657+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f0c6006e760 0x7f0c60070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.657+0000 7f0c7e59c700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0c70007d10 tx=0x7f0c70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.657+0000 7f0c7ed9d700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f0c6006e760 0x7f0c60070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.658+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== osd.2 v2:172.21.15.144:6800/103559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0c60071240 con 0x7f0c6006e760 2023-11-28T11:56:36.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 --> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0c8006a7a0 con 0x7f0c6006e760 2023-11-28T11:56:36.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c6effd700 1 -- 172.21.15.144:0/1070201153 <== osd.2 v2:172.21.15.144:6800/103559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0c8006a7a0 con 0x7f0c6006e760 2023-11-28T11:56:36.673 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:36.673 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:36.673 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:36.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f0c6006e760 msgr2=0x7f0c60070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6800/103559,v1:172.21.15.144:6801/103559] conn(0x7f0c6006e760 0x7f0c60070b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.674 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 msgr2=0x7f0c6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f0c600682f0 0x7f0c6006a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0c70007d10 tx=0x7f0c70009040 comp rx=0 tx=0).stop 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c800acb90 msgr2=0x7f0c8014cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 --2- 172.21.15.144:0/1070201153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c800acb90 0x7f0c8014cd30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400f610 tx=0x7f0c7400f9d0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c7ed9d700 1 -- 172.21.15.144:0/1070201153 reap_dead start 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 shutdown_connections 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 >> 172.21.15.144:0/1070201153 conn(0x7f0c8006b380 msgr2=0x7f0c800adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 shutdown_connections 2023-11-28T11:56:36.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.672+0000 7f0c84bc2700 1 -- 172.21.15.144:0/1070201153 wait complete. 2023-11-28T11:56:36.685 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T11:56:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.805+0000 7fa4a8d9e700 1 Processor -- start 2023-11-28T11:56:36.806 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.805+0000 7fa4a8d9e700 1 -- start start 2023-11-28T11:56:36.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a8d9e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a415cd30 0x7fa4a4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a8d9e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa4a406a980 0x7fa4a40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.808 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a8d9e700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40d0840 con 0x7fa4a40cfda0 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a8d9e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a40d0b20 con 0x7fa4a406a980 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a8d9e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4160670 con 0x7fa4a415cd30 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a1d9b700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa4a40cfda0 0x7fa4a406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56338/0 (socket says 172.21.15.144:56338) 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a1d9b700 1 -- 172.21.15.144:0/3856082181 learned_addr learned my addr 172.21.15.144:0/3856082181 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a2d9d700 1 --2- 172.21.15.144:0/3856082181 >> v2:172.21.15.177:3300/0 conn(0x7fa4a406a980 0x7fa4a40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a259c700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a415cd30 0x7fa4a4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.809 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a159a700 1 -- 172.21.15.144:0/3856082181 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705940780 0 0) 0x7fa4a40d0840 con 0x7fa4a40cfda0 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.806+0000 7fa4a159a700 1 -- 172.21.15.144:0/3856082181 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa48c0035b0 con 0x7fa4a40cfda0 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a159a700 1 -- 172.21.15.144:0/3856082181 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3746866650 0 0) 0x7fa48c0035b0 con 0x7fa4a40cfda0 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a415cd30 msgr2=0x7fa4a4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a2d9d700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a415cd30 0x7fa4a4161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/3856082181 >> v1:172.21.15.144:6789/0 conn(0x7fa4a40cfda0 legacy=0x7fa4a406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/3856082181 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4160950 con 0x7fa4a406a980 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a259c700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a415cd30 0x7fa4a4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:36.810 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a2d9d700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40d0210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009d90 tx=0x7fa49c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebe4022c5d092a77 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a159a700 1 -- 172.21.15.144:0/3856082181 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00e070 con 0x7fa4a406a980 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a159a700 1 -- 172.21.15.144:0/3856082181 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c003a20 con 0x7fa4a406a980 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.807+0000 7fa4a159a700 1 -- 172.21.15.144:0/3856082181 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494001670 con 0x7fa4a406a980 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 msgr2=0x7fa4a40d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40d0210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009d90 tx=0x7fa49c00b330 comp rx=0 tx=0).stop 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/3856082181 shutdown_connections 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40d0210 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.811 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/3856082181 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a415cd30 0x7fa4a4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/3856082181 >> 172.21.15.144:0/3856082181 conn(0x7fa4a406b380 msgr2=0x7fa4a406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:36.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/3856082181 shutdown_connections 2023-11-28T11:56:36.812 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.808+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/3856082181 wait complete. 2023-11-28T11:56:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.809+0000 7fa4a8d9e700 1 Processor -- start 2023-11-28T11:56:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.809+0000 7fa4a8d9e700 1 -- start start 2023-11-28T11:56:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a8d9e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a8d9e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a40cfda0 0x7fa4a40c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a8d9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 0x7fa4a415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.815 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a8d9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40c6760 con 0x7fa4a415cd30 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a8d9e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40b3f50 con 0x7fa4a406a980 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a8d9e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40b3340 con 0x7fa4a40cfda0 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 0x7fa4a415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 0x7fa4a415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39634/0 (socket says 172.21.15.144:39634) 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/968411190 learned_addr learned my addr 172.21.15.144:0/968411190 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a259c700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a1d9b700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a40cfda0 0x7fa4a40c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.816 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a40cfda0 msgr2=0x7fa4a40c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a40cfda0 0x7fa4a40c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 msgr2=0x7fa4a40b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 -- 172.21.15.144:0/968411190 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980094a0 con 0x7fa4a415cd30 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a1d9b700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a40cfda0 0x7fa4a40c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa4a2d9d700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 0x7fa4a415a1c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4a406edf0 tx=0x7fa49c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.810+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c009410 con 0x7fa4a415cd30 2023-11-28T11:56:36.817 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.811+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c004590 con 0x7fa4a415cd30 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.811+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c009a70 con 0x7fa4a415cd30 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.811+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c004940 con 0x7fa4a415cd30 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.811+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40cc280 con 0x7fa4a415cd30 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.812+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a4160c30 con 0x7fa4a415cd30 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.812+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa49c0124d0 con 0x7fa4a415cd30 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.812+0000 7fa492ffd700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa4840682f0 0x7fa48406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.812+0000 7fa4a259c700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa4840682f0 0x7fa48406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:36.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.812+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa49c087f50 con 0x7fa4a415cd30 2023-11-28T11:56:36.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.813+0000 7fa4a259c700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa4840682f0 0x7fa48406a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa498009c60 tx=0x7fa498008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:36.819 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.817+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa49c054a50 con 0x7fa4a415cd30 2023-11-28T11:56:36.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.998+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4a4070de0 con 0x7fa4840682f0 2023-11-28T11:56:36.999 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:36.999+0000 7fa492ffd700 1 -- 172.21.15.144:0/968411190 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa4a4070de0 con 0x7fa4840682f0 2023-11-28T11:56:37.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa4840682f0 msgr2=0x7fa48406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa4840682f0 0x7fa48406a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa498009c60 tx=0x7fa498008040 comp rx=0 tx=0).stop 2023-11-28T11:56:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 msgr2=0x7fa4a415a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 0x7fa4a415a1c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4a406edf0 tx=0x7fa49c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 shutdown_connections 2023-11-28T11:56:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4a415cd30 0x7fa4a415a1c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fa4840682f0 0x7fa48406a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4a406a980 0x7fa4a40b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 --2- 172.21.15.144:0/968411190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4a40cfda0 0x7fa4a40c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 >> 172.21.15.144:0/968411190 conn(0x7fa4a406b380 msgr2=0x7fa4a406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 shutdown_connections 2023-11-28T11:56:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.001+0000 7fa4a8d9e700 1 -- 172.21.15.144:0/968411190 wait complete. 2023-11-28T11:56:37.003 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T11:56:37.013 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T11:56:37.014 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:37.138 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.138+0000 7f80b3c33700 1 Processor -- start 2023-11-28T11:56:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.138+0000 7f80b3c33700 1 -- start start 2023-11-28T11:56:37.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b3c33700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f80ac0d0470 0x7f80ac071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b3c33700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac0cc7b0 con 0x7f80ac1596b0 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b3c33700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac0cd610 con 0x7f80ac0cc260 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b3c33700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ac159a80 con 0x7f80ac0d0470 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b19cf700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f80ac1596b0 0x7f80ac0cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56350/0 (socket says 172.21.15.144:56350) 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b19cf700 1 -- 172.21.15.144:0/4285226250 learned_addr learned my addr 172.21.15.144:0/4285226250 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b11ce700 1 --2- 172.21.15.144:0/4285226250 >> v2:172.21.15.177:3300/0 conn(0x7f80ac0d0470 0x7f80ac071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400454945 0 0) 0x7f80ac0cc7b0 con 0x7f80ac1596b0 2023-11-28T11:56:37.143 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80980035b0 con 0x7f80ac1596b0 2023-11-28T11:56:37.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.139+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30472599 0 0) 0x7f80980035b0 con 0x7f80ac1596b0 2023-11-28T11:56:37.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 >> v2:172.21.15.177:3300/0 conn(0x7f80ac0d0470 msgr2=0x7f80ac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 --2- 172.21.15.144:0/4285226250 >> v2:172.21.15.177:3300/0 conn(0x7f80ac0d0470 0x7f80ac071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 >> v1:172.21.15.177:6789/0 conn(0x7f80ac0cc260 legacy=0x7f80ac06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0cca00 con 0x7f80ac1596b0 2023-11-28T11:56:37.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f809c001670 con 0x7f80ac1596b0 2023-11-28T11:56:37.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b11ce700 1 --2- 172.21.15.144:0/4285226250 >> v2:172.21.15.177:3300/0 conn(0x7f80ac0d0470 0x7f80ac071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:56:37.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 >> v1:172.21.15.144:6789/0 conn(0x7f80ac1596b0 legacy=0x7f80ac0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f8098003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80980043f0 0x7f80980048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80980050b0 0x7f809800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac159a80 con 0x7f8098003a00 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809800c000 con 0x7f80980050b0 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809800e3d0 con 0x7f80980043f0 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f8098003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b11ce700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80980043f0 0x7f80980048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b21d0700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80980050b0 0x7f809800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 -- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80980043f0 msgr2=0x7f80980048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.146 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80980043f0 0x7f80980048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 -- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80980050b0 msgr2=0x7f809800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80980050b0 0x7f809800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 -- 172.21.15.144:0/4285226250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980035b0 con 0x7f8098003a00 2023-11-28T11:56:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b21d0700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80980050b0 0x7f809800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b21d0700 1 -- 172.21.15.144:0/4285226250 reap_dead start 2023-11-28T11:56:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b19cf700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f8098003dd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c2f0 tx=0x7f8098005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9eada176b88954 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c00ce30 con 0x7f8098003a00 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c005960 con 0x7f8098003a00 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.140+0000 7f80b09cd700 1 -- 172.21.15.144:0/4285226250 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c005d10 con 0x7f8098003a00 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 -- 172.21.15.144:0/4285226250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 msgr2=0x7f8098003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f8098003dd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c2f0 tx=0x7f8098005880 comp rx=0 tx=0).stop 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 -- 172.21.15.144:0/4285226250 shutdown_connections 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 --2- 172.21.15.144:0/4285226250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f8098003dd0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.148 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 -- 172.21.15.144:0/4285226250 >> 172.21.15.144:0/4285226250 conn(0x7f80ac06b380 msgr2=0x7f80ac06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 -- 172.21.15.144:0/4285226250 shutdown_connections 2023-11-28T11:56:37.149 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.141+0000 7f80b3c33700 1 -- 172.21.15.144:0/4285226250 wait complete. 2023-11-28T11:56:37.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.142+0000 7f80b3c33700 1 Processor -- start 2023-11-28T11:56:37.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.142+0000 7f80b3c33700 1 -- start start 2023-11-28T11:56:37.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b3c33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f80ac0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b3c33700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac15be00 0x7f80ac15c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.152 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b3c33700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac159ce0 0x7f80ac15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b3c33700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac0c8260 con 0x7f8098003a00 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b3c33700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac0c7690 con 0x7f80ac15be00 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b3c33700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac0c7930 con 0x7f80ac159ce0 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f80ac0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f80ac0b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39652/0 (socket says 172.21.15.144:39652) 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 -- 172.21.15.144:0/44295609 learned_addr learned my addr 172.21.15.144:0/44295609 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b11ce700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac15be00 0x7f80ac15c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.153 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b21d0700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac159ce0 0x7f80ac15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 -- 172.21.15.144:0/44295609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac159ce0 msgr2=0x7f80ac15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80ac159ce0 0x7f80ac15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 -- 172.21.15.144:0/44295609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac15be00 msgr2=0x7f80ac15c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80ac15be00 0x7f80ac15c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 -- 172.21.15.144:0/44295609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0ccfc0 con 0x7f8098003a00 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.143+0000 7f80b19cf700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f80ac0b1fd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f809c008fd0 tx=0x7f809c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.144+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c007c20 con 0x7f8098003a00 2023-11-28T11:56:37.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.144+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c004870 con 0x7f8098003a00 2023-11-28T11:56:37.155 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.144+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac0ccce0 con 0x7f8098003a00 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.144+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c004c00 con 0x7f8098003a00 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.144+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac0cd2a0 con 0x7f8098003a00 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.144+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80ac068690 con 0x7f8098003a00 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.145+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f809c010070 con 0x7f8098003a00 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.145+0000 7f80a27fc700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80900682f0 0x7f809006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.145+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f809c087390 con 0x7f8098003a00 2023-11-28T11:56:37.156 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.145+0000 7f80b11ce700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80900682f0 0x7f809006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.146+0000 7f80a27fc700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f809006e760 0x7f8090070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.146+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 --> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8090071240 con 0x7f809006e760 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.146+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f809c0877d0 con 0x7f8098003a00 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.146+0000 7f80b21d0700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f809006e760 0x7f8090070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.146+0000 7f80b11ce700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80900682f0 0x7f809006a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0c3650 tx=0x7f80a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.146+0000 7f80b21d0700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f809006e760 0x7f8090070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.157 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.147+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== osd.3 v2:172.21.15.144:6823/103575 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8090071240 con 0x7f809006e760 2023-11-28T11:56:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.160+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 --> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f80ac0c83a0 con 0x7f809006e760 2023-11-28T11:56:37.161 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80a27fc700 1 -- 172.21.15.144:0/44295609 <== osd.3 v2:172.21.15.144:6823/103575 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f80a4001ba0 con 0x7f809006e760 2023-11-28T11:56:37.161 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:37.162 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:37.162 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:37.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f809006e760 msgr2=0x7f8090070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6823/103575,v1:172.21.15.144:6825/103575] conn(0x7f809006e760 0x7f8090070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80900682f0 msgr2=0x7f809006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80900682f0 0x7f809006a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0c3650 tx=0x7f80a8008040 comp rx=0 tx=0).stop 2023-11-28T11:56:37.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 msgr2=0x7f80ac0b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.163 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 --2- 172.21.15.144:0/44295609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8098003a00 0x7f80ac0b1fd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f809c008fd0 tx=0x7f809c001a00 comp rx=0 tx=0).stop 2023-11-28T11:56:37.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b21d0700 1 -- 172.21.15.144:0/44295609 reap_dead start 2023-11-28T11:56:37.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 shutdown_connections 2023-11-28T11:56:37.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 >> 172.21.15.144:0/44295609 conn(0x7f80ac06b380 msgr2=0x7f80ac0cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 shutdown_connections 2023-11-28T11:56:37.164 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.161+0000 7f80b3c33700 1 -- 172.21.15.144:0/44295609 wait complete. 2023-11-28T11:56:37.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T11:56:37.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.294+0000 7f61fe153700 1 Processor -- start 2023-11-28T11:56:37.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.294+0000 7f61fe153700 1 -- start start 2023-11-28T11:56:37.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.295+0000 7f61fe153700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f61f81658c0 0x7f61f8167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.295+0000 7f61fe153700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f815d060 con 0x7f61f806dee0 2023-11-28T11:56:37.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.295+0000 7f61fe153700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f815d340 con 0x7f61f8071040 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.295+0000 7f61fe153700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f8161550 con 0x7f61f81658c0 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.295+0000 7f61f6ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f61f806dee0 0x7f61f806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56372/0 (socket says 172.21.15.144:56372) 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.295+0000 7f61f6ffd700 1 -- 172.21.15.144:0/436652032 learned_addr learned my addr 172.21.15.144:0/436652032 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947815978 0 0) 0x7f61f815d060 con 0x7f61f806dee0 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f7fff700 1 --2- 172.21.15.144:0/436652032 >> v2:172.21.15.177:3300/0 conn(0x7f61f81658c0 0x7f61f8167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e40035b0 con 0x7f61f806dee0 2023-11-28T11:56:37.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017893791 0 0) 0x7f61e40035b0 con 0x7f61f806dee0 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 >> v2:172.21.15.177:3300/0 conn(0x7f61f81658c0 msgr2=0x7f61f8167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 --2- 172.21.15.144:0/436652032 >> v2:172.21.15.177:3300/0 conn(0x7f61f81658c0 0x7f61f8167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 >> v1:172.21.15.177:6789/0 conn(0x7f61f8071040 legacy=0x7f61f8071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f815d620 con 0x7f61f806dee0 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f61e0001670 con 0x7f61f806dee0 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 >> v1:172.21.15.144:6789/0 conn(0x7f61f806dee0 legacy=0x7f61f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4003a00 0x7f61e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e4004850 0x7f61e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61e4005320 0x7f61e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8161550 con 0x7f61e4004850 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e400c250 con 0x7f61e4005320 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e400c410 con 0x7f61e4003a00 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f6ffd700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e4004850 0x7f61e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f77fe700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4003a00 0x7f61e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f7fff700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61e4005320 0x7f61e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f6ffd700 1 -- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4003a00 msgr2=0x7f61e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f6ffd700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4003a00 0x7f61e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f6ffd700 1 -- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61e4005320 msgr2=0x7f61e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f6ffd700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61e4005320 0x7f61e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f7fff700 1 -- 172.21.15.144:0/436652032 reap_dead start 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f6ffd700 1 -- 172.21.15.144:0/436652032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40035b0 con 0x7f61e4004850 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.296+0000 7f61f77fe700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4003a00 0x7f61e4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:37.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61f6ffd700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e4004850 0x7f61e4004cc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c240 tx=0x7f61e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab02525b9ed35d8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0007c10 con 0x7f61e4004850 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e000cd60 con 0x7f61e4004850 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61f67fc700 1 -- 172.21.15.144:0/436652032 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0005ba0 con 0x7f61e4004850 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 -- 172.21.15.144:0/436652032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e4004850 msgr2=0x7f61e4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e4004850 0x7f61e4004cc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f61e000c240 tx=0x7f61e4005880 comp rx=0 tx=0).stop 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 -- 172.21.15.144:0/436652032 shutdown_connections 2023-11-28T11:56:37.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 --2- 172.21.15.144:0/436652032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e4004850 0x7f61e4004cc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 -- 172.21.15.144:0/436652032 >> 172.21.15.144:0/436652032 conn(0x7f61f80d0000 msgr2=0x7f61f80d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 -- 172.21.15.144:0/436652032 shutdown_connections 2023-11-28T11:56:37.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.297+0000 7f61fe153700 1 -- 172.21.15.144:0/436652032 wait complete. 2023-11-28T11:56:37.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.298+0000 7f61fe153700 1 Processor -- start 2023-11-28T11:56:37.305 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 -- start start 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4004850 0x7f61f80b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f81587f0 0x7f61f80c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 0x7f61f80ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b3320 con 0x7f61f80c61f0 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81f6880 con 0x7f61f81587f0 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61fe153700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f81f5d00 con 0x7f61e4004850 2023-11-28T11:56:37.308 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 0x7f61f80ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 0x7f61f80ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39676/0 (socket says 172.21.15.144:39676) 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.299+0000 7f61f7fff700 1 -- 172.21.15.144:0/3850103229 learned_addr learned my addr 172.21.15.144:0/3850103229 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f6ffd700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f81587f0 0x7f61f80c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f77fe700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4004850 0x7f61f80b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f7fff700 1 -- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4004850 msgr2=0x7f61f80b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f7fff700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4004850 0x7f61f80b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f7fff700 1 -- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f81587f0 msgr2=0x7f61f80c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f7fff700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f81587f0 0x7f61f80c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f7fff700 1 -- 172.21.15.144:0/3850103229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f815ccd0 con 0x7f61f80c61f0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61f7fff700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 0x7f61f80ae1d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f61f8071d40 tx=0x7f61f0011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000e040 con 0x7f61f80c61f0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f000f040 con 0x7f61f80c61f0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f815d900 con 0x7f61f80c61f0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f000fa00 con 0x7f61f80c61f0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.300+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f806a270 con 0x7f61f80c61f0 2023-11-28T11:56:37.310 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.301+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8150a00 con 0x7f61f80c61f0 2023-11-28T11:56:37.311 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.301+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f61f0025070 con 0x7f61f80c61f0 2023-11-28T11:56:37.311 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.302+0000 7f61effff700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f61d80682f0 0x7f61d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.311 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.302+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f61f0086120 con 0x7f61f80c61f0 2023-11-28T11:56:37.311 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.302+0000 7f61f77fe700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f61d80682f0 0x7f61d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.311 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.302+0000 7f61f77fe700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f61d80682f0 0x7f61d806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000c00 tx=0x7f61e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.311 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.306+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61f8150a00 con 0x7f61f80c61f0 2023-11-28T11:56:37.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.488+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f8150a00 con 0x7f61d80682f0 2023-11-28T11:56:37.489 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.489+0000 7f61effff700 1 -- 172.21.15.144:0/3850103229 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f61f8150a00 con 0x7f61d80682f0 2023-11-28T11:56:37.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f61d80682f0 msgr2=0x7f61d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f61d80682f0 0x7f61d806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000c00 tx=0x7f61e8009040 comp rx=0 tx=0).stop 2023-11-28T11:56:37.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 msgr2=0x7f61f80ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 0x7f61f80ae1d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f61f8071d40 tx=0x7f61f0011f80 comp rx=0 tx=0).stop 2023-11-28T11:56:37.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 shutdown_connections 2023-11-28T11:56:37.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61f80c61f0 0x7f61f80ae1d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f61d80682f0 0x7f61d806a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61e4004850 0x7f61f80b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 --2- 172.21.15.144:0/3850103229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61f81587f0 0x7f61f80c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 >> 172.21.15.144:0/3850103229 conn(0x7f61f80d0000 msgr2=0x7f61f815f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 shutdown_connections 2023-11-28T11:56:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.491+0000 7f61fe153700 1 -- 172.21.15.144:0/3850103229 wait complete. 2023-11-28T11:56:37.494 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T11:56:37.505 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T11:56:37.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:37.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.629+0000 7fd6184f2700 1 Processor -- start 2023-11-28T11:56:37.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.630+0000 7fd6184f2700 1 -- start start 2023-11-28T11:56:37.640 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.630+0000 7fd6184f2700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6100b0190 0x7fd61006f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.630+0000 7fd6184f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd61006c710 0x7fd61006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.630+0000 7fd6184f2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61006d080 con 0x7fd6100b28e0 2023-11-28T11:56:37.641 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.630+0000 7fd6184f2700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6100cff80 con 0x7fd6100b0190 2023-11-28T11:56:37.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.630+0000 7fd6184f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100d0260 con 0x7fd61006c710 2023-11-28T11:56:37.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6100b0190 0x7fd61006f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6100b0190 0x7fd61006f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39680/0 (socket says 172.21.15.144:39680) 2023-11-28T11:56:37.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 -- 172.21.15.144:0/2446994846 learned_addr learned my addr 172.21.15.144:0/2446994846 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:37.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd615a8d700 1 --2- 172.21.15.144:0/2446994846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd61006c710 0x7fd61006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 -- 172.21.15.144:0/2446994846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd61006c710 msgr2=0x7fd61006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 --2- 172.21.15.144:0/2446994846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd61006c710 0x7fd61006cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.643 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 -- 172.21.15.144:0/2446994846 >> v1:172.21.15.177:6789/0 conn(0x7fd6100b28e0 legacy=0x7fd6100b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 -- 172.21.15.144:0/2446994846 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100d0540 con 0x7fd6100b0190 2023-11-28T11:56:37.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61628e700 1 --2- 172.21.15.144:0/2446994846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6100b0190 0x7fd61006f7f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd6000098f0 tx=0x7fd600009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190a23576d045921 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61528c700 1 -- 172.21.15.144:0/2446994846 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600011830 con 0x7fd6100b0190 2023-11-28T11:56:37.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61528c700 1 -- 172.21.15.144:0/2446994846 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600011990 con 0x7fd6100b0190 2023-11-28T11:56:37.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.631+0000 7fd61528c700 1 -- 172.21.15.144:0/2446994846 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600011d20 con 0x7fd6100b0190 2023-11-28T11:56:37.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 -- 172.21.15.144:0/2446994846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6100b0190 msgr2=0x7fd61006f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 --2- 172.21.15.144:0/2446994846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6100b0190 0x7fd61006f7f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd6000098f0 tx=0x7fd600009c00 comp rx=0 tx=0).stop 2023-11-28T11:56:37.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 -- 172.21.15.144:0/2446994846 shutdown_connections 2023-11-28T11:56:37.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 --2- 172.21.15.144:0/2446994846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6100b0190 0x7fd61006f7f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 --2- 172.21.15.144:0/2446994846 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd61006c710 0x7fd61006cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 -- 172.21.15.144:0/2446994846 >> 172.21.15.144:0/2446994846 conn(0x7fd61006aec0 msgr2=0x7fd61006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 -- 172.21.15.144:0/2446994846 shutdown_connections 2023-11-28T11:56:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.632+0000 7fd6184f2700 1 -- 172.21.15.144:0/2446994846 wait complete. 2023-11-28T11:56:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.633+0000 7fd6184f2700 1 Processor -- start 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.633+0000 7fd6184f2700 1 -- start start 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6100b0190 0x7fd610150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6100b28e0 0x7fd6101e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61015fa10 0x7fd6101e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100b6bf0 con 0x7fd61015fa10 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6101e6b60 con 0x7fd6100b28e0 2023-11-28T11:56:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6101e6e40 con 0x7fd6100b0190 2023-11-28T11:56:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61015fa10 0x7fd6101e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61015fa10 0x7fd6101e7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39688/0 (socket says 172.21.15.144:39688) 2023-11-28T11:56:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 -- 172.21.15.144:0/3671675565 learned_addr learned my addr 172.21.15.144:0/3671675565 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd615a8d700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6100b28e0 0x7fd6101e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd61628e700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6100b0190 0x7fd610150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 -- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6100b0190 msgr2=0x7fd610150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6100b0190 0x7fd610150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 -- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6100b28e0 msgr2=0x7fd6101e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6100b28e0 0x7fd6101e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 -- 172.21.15.144:0/3671675565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600009550 con 0x7fd61015fa10 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd616a8f700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61015fa10 0x7fd6101e7c00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd60800cc50 tx=0x7fd60800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608003c00 con 0x7fd61015fa10 2023-11-28T11:56:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608003d60 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd610161330 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.634+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800aab0 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.635+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6100d0820 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd6100cfbe0 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd608010050 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd606ffd700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5f40682f0 0x7fd5f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd6080857d0 con 0x7fd61015fa10 2023-11-28T11:56:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd61628e700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5f40682f0 0x7fd5f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd606ffd700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fd5f406e760 0x7fd5f4070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 --> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5f4071240 con 0x7fd5f406e760 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.636+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd60808a2a0 con 0x7fd61015fa10 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.637+0000 7fd615a8d700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fd5f406e760 0x7fd5f4070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.637+0000 7fd61628e700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5f40682f0 0x7fd5f406a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd600009c40 tx=0x7fd600008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.637+0000 7fd615a8d700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fd5f406e760 0x7fd5f4070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.638+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== osd.4 v2:172.21.15.177:6808/103772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd5f4071240 con 0x7fd5f406e760 2023-11-28T11:56:37.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.652+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 --> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd610068690 con 0x7fd5f406e760 2023-11-28T11:56:37.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.652+0000 7fd606ffd700 1 -- 172.21.15.144:0/3671675565 <== osd.4 v2:172.21.15.177:6808/103772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd610068690 con 0x7fd5f406e760 2023-11-28T11:56:37.656 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:37.656 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:37.656 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:37.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fd5f406e760 msgr2=0x7fd5f4070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.177:6808/103772,v1:172.21.15.177:6809/103772] conn(0x7fd5f406e760 0x7fd5f4070b70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5f40682f0 msgr2=0x7fd5f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7fd5f40682f0 0x7fd5f406a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd600009c40 tx=0x7fd600008030 comp rx=0 tx=0).stop 2023-11-28T11:56:37.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61015fa10 msgr2=0x7fd6101e7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 --2- 172.21.15.144:0/3671675565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd61015fa10 0x7fd6101e7c00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fd60800cc50 tx=0x7fd60800cf60 comp rx=0 tx=0).stop 2023-11-28T11:56:37.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd616a8f700 1 -- 172.21.15.144:0/3671675565 reap_dead start 2023-11-28T11:56:37.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 shutdown_connections 2023-11-28T11:56:37.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 >> 172.21.15.144:0/3671675565 conn(0x7fd61006c710 msgr2=0x7fd610160b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 shutdown_connections 2023-11-28T11:56:37.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.653+0000 7fd6184f2700 1 -- 172.21.15.144:0/3671675565 wait complete. 2023-11-28T11:56:37.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T11:56:37.787 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.787+0000 7f7a1eca6700 1 Processor -- start 2023-11-28T11:56:37.789 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.787+0000 7f7a1eca6700 1 -- start start 2023-11-28T11:56:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1eca6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a1806ec90 0x7f7a18065ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1eca6700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a180cb0e0 0x7f7a1806d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1eca6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a18066580 con 0x7f7a180cec70 2023-11-28T11:56:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1eca6700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a180cded0 con 0x7f7a180cb0e0 2023-11-28T11:56:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1eca6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180ce1b0 con 0x7f7a1806ec90 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a180cb0e0 0x7f7a1806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a180cb0e0 0x7f7a1806d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39698/0 (socket says 172.21.15.144:39698) 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 -- 172.21.15.144:0/428112027 learned_addr learned my addr 172.21.15.144:0/428112027 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1ca42700 1 --2- 172.21.15.144:0/428112027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a1806ec90 0x7f7a18065ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 -- 172.21.15.144:0/428112027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a1806ec90 msgr2=0x7f7a18065ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 --2- 172.21.15.144:0/428112027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a1806ec90 0x7f7a18065ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.794 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 -- 172.21.15.144:0/428112027 >> v1:172.21.15.177:6789/0 conn(0x7f7a180cec70 legacy=0x7f7a180ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 -- 172.21.15.144:0/428112027 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180ce490 con 0x7f7a180cb0e0 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a1d243700 1 --2- 172.21.15.144:0/428112027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a180cb0e0 0x7f7a1806d230 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009e50 tx=0x7f7a1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f186234f74eea76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a177fe700 1 -- 172.21.15.144:0/428112027 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000e070 con 0x7f7a180cb0e0 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a177fe700 1 -- 172.21.15.144:0/428112027 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a10003a20 con 0x7f7a180cb0e0 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.788+0000 7f7a177fe700 1 -- 172.21.15.144:0/428112027 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10003dd0 con 0x7f7a180cb0e0 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 -- 172.21.15.144:0/428112027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a180cb0e0 msgr2=0x7f7a1806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/428112027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a180cb0e0 0x7f7a1806d230 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009e50 tx=0x7f7a1000b330 comp rx=0 tx=0).stop 2023-11-28T11:56:37.795 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 -- 172.21.15.144:0/428112027 shutdown_connections 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/428112027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a1806ec90 0x7f7a18065ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/428112027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a180cb0e0 0x7f7a1806d230 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 -- 172.21.15.144:0/428112027 >> 172.21.15.144:0/428112027 conn(0x7f7a180b12b0 msgr2=0x7f7a180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 -- 172.21.15.144:0/428112027 shutdown_connections 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.789+0000 7f7a1eca6700 1 -- 172.21.15.144:0/428112027 wait complete. 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.790+0000 7f7a1eca6700 1 Processor -- start 2023-11-28T11:56:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.790+0000 7f7a1eca6700 1 -- start start 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1eca6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 0x7f7a181f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1eca6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a180cb0e0 0x7f7a180b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1eca6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a180cec70 0x7f7a1814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1eca6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180a8c00 con 0x7f7a1806ec90 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1eca6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180c6b60 con 0x7f7a180cec70 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1eca6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a18159a60 con 0x7f7a180cb0e0 2023-11-28T11:56:37.797 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 0x7f7a181f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 0x7f7a181f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39700/0 (socket says 172.21.15.144:39700) 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 -- 172.21.15.144:0/2735968424 learned_addr learned my addr 172.21.15.144:0/2735968424 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a17fff700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a180cb0e0 0x7f7a180b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1d243700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a180cec70 0x7f7a1814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 -- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a180cb0e0 msgr2=0x7f7a180b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a180cb0e0 0x7f7a180b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.798 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 -- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a180cec70 msgr2=0x7f7a1814d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a180cec70 0x7f7a1814d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 -- 172.21.15.144:0/2735968424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a10009b30 con 0x7f7a1806ec90 2023-11-28T11:56:37.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.791+0000 7f7a1ca42700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 0x7f7a181f44f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c7f0 tx=0x7f7a0c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.792+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c011800 con 0x7f7a1806ec90 2023-11-28T11:56:37.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.792+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c011960 con 0x7f7a1806ec90 2023-11-28T11:56:37.799 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.792+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a180cb6c0 con 0x7f7a1806ec90 2023-11-28T11:56:37.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.792+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c011cd0 con 0x7f7a1806ec90 2023-11-28T11:56:37.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.792+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a180ce770 con 0x7f7a1806ec90 2023-11-28T11:56:37.800 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.793+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1806f060 con 0x7f7a1806ec90 2023-11-28T11:56:37.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.793+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a0c017030 con 0x7f7a1806ec90 2023-11-28T11:56:37.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.793+0000 7f7a157fa700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79fc0682f0 0x7f79fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:37.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.793+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7a0c0853c0 con 0x7f7a1806ec90 2023-11-28T11:56:37.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.794+0000 7f7a17fff700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79fc0682f0 0x7f79fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:37.801 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.794+0000 7f7a17fff700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79fc0682f0 0x7f79fc06a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a1804e760 tx=0x7f7a04007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:37.802 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.798+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a0c051ea0 con 0x7f7a1806ec90 2023-11-28T11:56:37.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.979+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a180cf110 con 0x7f79fc0682f0 2023-11-28T11:56:37.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.979+0000 7f7a157fa700 1 -- 172.21.15.144:0/2735968424 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7a180cf110 con 0x7f79fc0682f0 2023-11-28T11:56:37.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79fc0682f0 msgr2=0x7f79fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79fc0682f0 0x7f79fc06a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7a1804e760 tx=0x7f7a04007ec0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 msgr2=0x7f7a181f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:37.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 0x7f7a181f44f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c7f0 tx=0x7f7a0c00cb00 comp rx=0 tx=0).stop 2023-11-28T11:56:37.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 shutdown_connections 2023-11-28T11:56:37.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a1806ec90 0x7f7a181f44f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f79fc0682f0 0x7f79fc06a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a180cec70 0x7f7a1814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 --2- 172.21.15.144:0/2735968424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a180cb0e0 0x7f7a180b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:37.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 >> 172.21.15.144:0/2735968424 conn(0x7f7a180b12b0 msgr2=0x7f7a180cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:37.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 shutdown_connections 2023-11-28T11:56:37.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:37.981+0000 7f7a1eca6700 1 -- 172.21.15.144:0/2735968424 wait complete. 2023-11-28T11:56:37.984 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T11:56:37.994 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T11:56:37.994 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:38.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.119+0000 7f3c3776f700 1 Processor -- start 2023-11-28T11:56:38.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.119+0000 7f3c3776f700 1 -- start start 2023-11-28T11:56:38.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c3776f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3c3006a980 0x7f3c30070d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c3776f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c3006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c3776f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c300713c0 con 0x7f3c300af060 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c3776f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3006dee0 con 0x7f3c3006a980 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c3776f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300716a0 con 0x7f3c300af430 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c35d0c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c3006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c34d0a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3c3006a980 0x7f3c30070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c35d0c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c3006d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.144:51794/0 (socket says 172.21.15.144:51794) 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c35d0c700 1 -- 172.21.15.144:0/1649297869 learned_addr learned my addr 172.21.15.144:0/1649297869 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.123 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c3550b700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f3c300af060 0x7f3c3006a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.144:56910/0 (socket says 172.21.15.144:56910) 2023-11-28T11:56:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c34d0a700 1 -- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 msgr2=0x7f3c3006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c34d0a700 1 --2- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c3006d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c34d0a700 1 -- 172.21.15.144:0/1649297869 >> v1:172.21.15.177:6789/0 conn(0x7f3c300af060 legacy=0x7f3c3006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c34d0a700 1 -- 172.21.15.144:0/1649297869 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c30071980 con 0x7f3c3006a980 2023-11-28T11:56:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.120+0000 7f3c27fff700 1 -- 172.21.15.144:0/1649297869 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350910615 0 0) 0x7f3c300713c0 con 0x7f3c300af060 2023-11-28T11:56:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c34d0a700 1 --2- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c3006a980 0x7f3c30070d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009840 tx=0x7f3c2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7f22864f6af1c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c27fff700 1 -- 172.21.15.144:0/1649297869 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011830 con 0x7f3c3006a980 2023-11-28T11:56:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c27fff700 1 -- 172.21.15.144:0/1649297869 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c011990 con 0x7f3c3006a980 2023-11-28T11:56:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c27fff700 1 -- 172.21.15.144:0/1649297869 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c011d20 con 0x7f3c3006a980 2023-11-28T11:56:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 -- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c3006a980 msgr2=0x7f3c30070d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 --2- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c3006a980 0x7f3c30070d50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009840 tx=0x7f3c2c009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 -- 172.21.15.144:0/1649297869 shutdown_connections 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 --2- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c3006d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 --2- 172.21.15.144:0/1649297869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c3006a980 0x7f3c30070d50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 -- 172.21.15.144:0/1649297869 >> 172.21.15.144:0/1649297869 conn(0x7f3c3006b380 msgr2=0x7f3c3006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 -- 172.21.15.144:0/1649297869 shutdown_connections 2023-11-28T11:56:38.126 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.121+0000 7f3c3776f700 1 -- 172.21.15.144:0/1649297869 wait complete. 2023-11-28T11:56:38.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 Processor -- start 2023-11-28T11:56:38.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 -- start start 2023-11-28T11:56:38.130 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3006a980 0x7f3c300c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c300af060 0x7f3c300c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c300cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c301f2480 con 0x7f3c3006a980 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c301f2740 con 0x7f3c300af060 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3776f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c301f2a20 con 0x7f3c300af430 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3550b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3006a980 0x7f3c300c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3550b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3006a980 0x7f3c300c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39714/0 (socket says 172.21.15.144:39714) 2023-11-28T11:56:38.131 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.123+0000 7f3c3550b700 1 -- 172.21.15.144:0/2459456084 learned_addr learned my addr 172.21.15.144:0/2459456084 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c35d0c700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c300cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3550b700 1 -- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 msgr2=0x7f3c300cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3550b700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c300cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3550b700 1 -- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c300af060 msgr2=0x7f3c300c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c34d0a700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c300af060 0x7f3c300c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3550b700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3c300af060 0x7f3c300c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3550b700 1 -- 172.21.15.144:0/2459456084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c0094a0 con 0x7f3c3006a980 2023-11-28T11:56:38.132 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c35d0c700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3c300af430 0x7f3c300cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3550b700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3006a980 0x7f3c300c77b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3c200011e0 tx=0x7f3c2000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20004c80 con 0x7f3c3006a980 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20004e00 con 0x7f3c3006a980 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c30071f40 con 0x7f3c3006a980 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20007310 con 0x7f3c3006a980 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.124+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c30071c60 con 0x7f3c3006a980 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.125+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c300ca660 con 0x7f3c3006a980 2023-11-28T11:56:38.133 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3c2001a020 con 0x7f3c3006a980 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c25ffb700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3c140682f0 0x7f3c1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3c20085660 con 0x7f3c3006a980 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c34d0a700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3c140682f0 0x7f3c1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c25ffb700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7f3c1406e760 0x7f3c14070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c14071240 con 0x7f3c1406e760 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c20087d10 con 0x7f3c3006a980 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c34d0a700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3c140682f0 0x7f3c1406a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009840 tx=0x7f3c2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.134 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.126+0000 7f3c35d0c700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7f3c1406e760 0x7f3c14070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.127+0000 7f3c35d0c700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7f3c1406e760 0x7f3c14070b70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.135 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.128+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== osd.5 v2:172.21.15.177:6817/103774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3c14071240 con 0x7f3c1406e760 2023-11-28T11:56:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.141+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 --> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3c301f3640 con 0x7f3c1406e760 2023-11-28T11:56:38.142 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c25ffb700 1 -- 172.21.15.144:0/2459456084 <== osd.5 v2:172.21.15.177:6817/103774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3c301f3640 con 0x7f3c1406e760 2023-11-28T11:56:38.143 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:38.143 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:38.143 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:38.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7f3c1406e760 msgr2=0x7f3c14070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c3776f700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.177:6817/103774,v1:172.21.15.177:6819/103774] conn(0x7f3c1406e760 0x7f3c14070b70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3c140682f0 msgr2=0x7f3c1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c3776f700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f3c140682f0 0x7f3c1406a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009840 tx=0x7f3c2c008030 comp rx=0 tx=0).stop 2023-11-28T11:56:38.144 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3006a980 msgr2=0x7f3c300c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.142+0000 7f3c3776f700 1 --2- 172.21.15.144:0/2459456084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c3006a980 0x7f3c300c77b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3c200011e0 tx=0x7f3c2000cae0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.143+0000 7f3c35d0c700 1 -- 172.21.15.144:0/2459456084 reap_dead start 2023-11-28T11:56:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.143+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 shutdown_connections 2023-11-28T11:56:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.143+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 >> 172.21.15.144:0/2459456084 conn(0x7f3c3006b380 msgr2=0x7f3c300cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.143+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 shutdown_connections 2023-11-28T11:56:38.145 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.143+0000 7f3c3776f700 1 -- 172.21.15.144:0/2459456084 wait complete. 2023-11-28T11:56:38.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T11:56:38.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.274+0000 7efe05c34700 1 Processor -- start 2023-11-28T11:56:38.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.274+0000 7efe05c34700 1 -- start start 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efe05c34700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe0006dee0 0x7efe0006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efe05c34700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe00160540 0x7efe001609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efe05c34700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0015d060 con 0x7efe00071040 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efe05c34700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0015d340 con 0x7efe0006dee0 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efe05c34700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015d620 con 0x7efe00160540 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efdfeffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe0006dee0 0x7efe0006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efdfeffd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe0006dee0 0x7efe0006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39726/0 (socket says 172.21.15.144:39726) 2023-11-28T11:56:38.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.275+0000 7efdfeffd700 1 -- 172.21.15.144:0/727768970 learned_addr learned my addr 172.21.15.144:0/727768970 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfffff700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe00160540 0x7efe001609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfeffd700 1 -- 172.21.15.144:0/727768970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe00160540 msgr2=0x7efe001609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfeffd700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe00160540 0x7efe001609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfeffd700 1 -- 172.21.15.144:0/727768970 >> v1:172.21.15.177:6789/0 conn(0x7efe00071040 legacy=0x7efe00071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfeffd700 1 -- 172.21.15.144:0/727768970 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0015d900 con 0x7efe0006dee0 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfffff700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe00160540 0x7efe001609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfeffd700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0006dee0 0x7efe0006e330 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098f0 tx=0x7efdf0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7483bf0098d43767 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.297 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfe7fc700 1 -- 172.21.15.144:0/727768970 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000b8a0 con 0x7efe0006dee0 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfe7fc700 1 -- 172.21.15.144:0/727768970 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000ba00 con 0x7efe0006dee0 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efdfe7fc700 1 -- 172.21.15.144:0/727768970 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000bd90 con 0x7efe0006dee0 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efe05c34700 1 -- 172.21.15.144:0/727768970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0006dee0 msgr2=0x7efe0006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.276+0000 7efe05c34700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0006dee0 0x7efe0006e330 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098f0 tx=0x7efdf0009c00 comp rx=0 tx=0).stop 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.277+0000 7efe05c34700 1 -- 172.21.15.144:0/727768970 shutdown_connections 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.277+0000 7efe05c34700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe00160540 0x7efe001609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.298 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.277+0000 7efe05c34700 1 --2- 172.21.15.144:0/727768970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe0006dee0 0x7efe0006e330 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.277+0000 7efe05c34700 1 -- 172.21.15.144:0/727768970 >> 172.21.15.144:0/727768970 conn(0x7efe000d0000 msgr2=0x7efe000d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.277+0000 7efe05c34700 1 -- 172.21.15.144:0/727768970 shutdown_connections 2023-11-28T11:56:38.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.277+0000 7efe05c34700 1 -- 172.21.15.144:0/727768970 wait complete. 2023-11-28T11:56:38.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.278+0000 7efe05c34700 1 Processor -- start 2023-11-28T11:56:38.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.278+0000 7efe05c34700 1 -- start start 2023-11-28T11:56:38.299 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe0006dee0 0x7efe001582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 0x7efe000c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe00160540 0x7efe000c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000b3d40 con 0x7efe00071040 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000c5430 con 0x7efe00160540 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000c48b0 con 0x7efe0006dee0 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 0x7efe000c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 0x7efe000c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39732/0 (socket says 172.21.15.144:39732) 2023-11-28T11:56:38.300 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 -- 172.21.15.144:0/3326853387 learned_addr learned my addr 172.21.15.144:0/3326853387 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfffff700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe00160540 0x7efe000c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdff7fe700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe0006dee0 0x7efe001582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 -- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe0006dee0 msgr2=0x7efe001582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe0006dee0 0x7efe001582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 -- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe00160540 msgr2=0x7efe000c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe00160540 0x7efe000c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 -- 172.21.15.144:0/3326853387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf0009550 con 0x7efe00071040 2023-11-28T11:56:38.301 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdff7fe700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe0006dee0 0x7efe001582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:38.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efdfeffd700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 0x7efe000c5f60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009c40 tx=0x7efdf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0004380 con 0x7efe00071040 2023-11-28T11:56:38.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf001e070 con 0x7efe00071040 2023-11-28T11:56:38.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0015ccd0 con 0x7efe00071040 2023-11-28T11:56:38.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00036a0 con 0x7efe00071040 2023-11-28T11:56:38.302 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.279+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0006e870 con 0x7efe00071040 2023-11-28T11:56:38.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.280+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0006a270 con 0x7efe00071040 2023-11-28T11:56:38.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.281+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efdf0029020 con 0x7efe00071040 2023-11-28T11:56:38.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.281+0000 7efde7fff700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efdec0682f0 0x7efdec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.281+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7efdf00869c0 con 0x7efe00071040 2023-11-28T11:56:38.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.281+0000 7efdff7fe700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efdec0682f0 0x7efdec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.303 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.281+0000 7efdff7fe700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efdec0682f0 0x7efdec06a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000f60 tx=0x7efdf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.304 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.285+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efdf0053520 con 0x7efe00071040 2023-11-28T11:56:38.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.466+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7efe00158830 con 0x7efdec0682f0 2023-11-28T11:56:38.467 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.466+0000 7efde7fff700 1 -- 172.21.15.144:0/3326853387 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efe00158830 con 0x7efdec0682f0 2023-11-28T11:56:38.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efdec0682f0 msgr2=0x7efdec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.469 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efdec0682f0 0x7efdec06a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000f60 tx=0x7efdf4009040 comp rx=0 tx=0).stop 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 msgr2=0x7efe000c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 0x7efe000c5f60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009c40 tx=0x7efdf0004750 comp rx=0 tx=0).stop 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 shutdown_connections 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7efdec0682f0 0x7efdec06a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe00160540 0x7efe000c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe0006dee0 0x7efe001582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.470 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 --2- 172.21.15.144:0/3326853387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe00071040 0x7efe000c5f60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.468+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 >> 172.21.15.144:0/3326853387 conn(0x7efe000d0000 msgr2=0x7efe0015e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.469+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 shutdown_connections 2023-11-28T11:56:38.471 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.469+0000 7efe05c34700 1 -- 172.21.15.144:0/3326853387 wait complete. 2023-11-28T11:56:38.471 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T11:56:38.481 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T11:56:38.481 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:38.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.605+0000 7f1a34d0d700 1 Processor -- start 2023-11-28T11:56:38.606 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.605+0000 7f1a34d0d700 1 -- start start 2023-11-28T11:56:38.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a34d0d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1a300cbea0 0x7f1a30071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.610 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a34d0d700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a300cd2e0 con 0x7f1a3015cd10 2023-11-28T11:56:38.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a34d0d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a300cc3f0 con 0x7f1a300ce3c0 2023-11-28T11:56:38.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a34d0d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a300cf250 con 0x7f1a300cbea0 2023-11-28T11:56:38.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2e59c700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1a3015cd10 0x7f1a300cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56450/0 (socket says 172.21.15.144:56450) 2023-11-28T11:56:38.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2e59c700 1 -- 172.21.15.144:0/3580327853 learned_addr learned my addr 172.21.15.144:0/3580327853 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:38.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2ed9d700 1 --2- 172.21.15.144:0/3580327853 >> v2:172.21.15.177:3300/0 conn(0x7f1a300cbea0 0x7f1a30071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977114824 0 0) 0x7f1a300cd2e0 con 0x7f1a3015cd10 2023-11-28T11:56:38.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a180035b0 con 0x7f1a3015cd10 2023-11-28T11:56:38.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 <== mon.0 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1311398646 0 0) 0x7f1a180035b0 con 0x7f1a3015cd10 2023-11-28T11:56:38.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 >> v2:172.21.15.177:3300/0 conn(0x7f1a300cbea0 msgr2=0x7f1a30071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 --2- 172.21.15.144:0/3580327853 >> v2:172.21.15.177:3300/0 conn(0x7f1a300cbea0 0x7f1a30071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 >> v1:172.21.15.177:6789/0 conn(0x7f1a300ce3c0 legacy=0x7f1a3006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300cc6d0 con 0x7f1a3015cd10 2023-11-28T11:56:38.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.606+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 <== mon.0 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4115049716 0 0) 0x7f1a24001670 con 0x7f1a3015cd10 2023-11-28T11:56:38.614 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 >> v1:172.21.15.144:6789/0 conn(0x7f1a3015cd10 legacy=0x7f1a300cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a18003a00 0x7f1a18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18004850 0x7f1a18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18005200 0x7f1a1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.615 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1800c160 con 0x7f1a18004850 2023-11-28T11:56:38.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1800c2d0 con 0x7f1a18003a00 2023-11-28T11:56:38.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1800c490 con 0x7f1a18005200 2023-11-28T11:56:38.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18004850 0x7f1a18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2e59c700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a18003a00 0x7f1a18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.616 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2ed9d700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18005200 0x7f1a1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 -- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18005200 msgr2=0x7f1a1800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18005200 0x7f1a1800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 -- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a18003a00 msgr2=0x7f1a18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a18003a00 0x7f1a18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.617 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 -- 172.21.15.144:0/3580327853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180035b0 con 0x7f1a18004850 2023-11-28T11:56:38.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2ed9d700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18005200 0x7f1a1800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:38.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2ed9d700 1 -- 172.21.15.144:0/3580327853 reap_dead start 2023-11-28T11:56:38.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18004850 0x7f1a18004cc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007fd0 tx=0x7f1a18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87021bbf1ad1183d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20001d80 con 0x7f1a18004850 2023-11-28T11:56:38.618 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a200052f0 con 0x7f1a18004850 2023-11-28T11:56:38.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.607+0000 7f1a2d59a700 1 -- 172.21.15.144:0/3580327853 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20001420 con 0x7f1a18004850 2023-11-28T11:56:38.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 -- 172.21.15.144:0/3580327853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18004850 msgr2=0x7f1a18004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18004850 0x7f1a18004cc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007fd0 tx=0x7f1a18005880 comp rx=0 tx=0).stop 2023-11-28T11:56:38.619 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 -- 172.21.15.144:0/3580327853 shutdown_connections 2023-11-28T11:56:38.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 --2- 172.21.15.144:0/3580327853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18004850 0x7f1a18004cc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 -- 172.21.15.144:0/3580327853 >> 172.21.15.144:0/3580327853 conn(0x7f1a3006b380 msgr2=0x7f1a3006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 -- 172.21.15.144:0/3580327853 shutdown_connections 2023-11-28T11:56:38.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.608+0000 7f1a34d0d700 1 -- 172.21.15.144:0/3580327853 wait complete. 2023-11-28T11:56:38.620 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.609+0000 7f1a34d0d700 1 Processor -- start 2023-11-28T11:56:38.621 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.609+0000 7f1a34d0d700 1 -- start start 2023-11-28T11:56:38.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a34d0d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18004850 0x7f1a301534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a34d0d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a300c6f10 0x7f1a300b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.624 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a34d0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a300c5c10 0x7f1a300c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a34d0d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a300c65c0 con 0x7f1a300c5c10 2023-11-28T11:56:38.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a34d0d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a301f5a10 con 0x7f1a300c6f10 2023-11-28T11:56:38.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a34d0d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a300c7380 con 0x7f1a18004850 2023-11-28T11:56:38.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a300c5c10 0x7f1a300c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.625 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a300c5c10 0x7f1a300c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39752/0 (socket says 172.21.15.144:39752) 2023-11-28T11:56:38.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 -- 172.21.15.144:0/2998498420 learned_addr learned my addr 172.21.15.144:0/2998498420 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2e59c700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18004850 0x7f1a301534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a300c6f10 0x7f1a300b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 -- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18004850 msgr2=0x7f1a301534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.626 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18004850 0x7f1a301534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 -- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a300c6f10 msgr2=0x7f1a300b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a300c6f10 0x7f1a300b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 -- 172.21.15.144:0/2998498420 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300ccc90 con 0x7f1a300c5c10 2023-11-28T11:56:38.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2e59c700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a18004850 0x7f1a301534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:38.627 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a300c6f10 0x7f1a300b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:38.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.610+0000 7f1a2ed9d700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a300c5c10 0x7f1a300c6080 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1a30070b90 tx=0x7f1a280104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.611+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2801b070 con 0x7f1a300c5c10 2023-11-28T11:56:38.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.611+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2800f040 con 0x7f1a300c5c10 2023-11-28T11:56:38.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.611+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a300cc9b0 con 0x7f1a300c5c10 2023-11-28T11:56:38.628 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.611+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800e050 con 0x7f1a300c5c10 2023-11-28T11:56:38.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.611+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a300ccf70 con 0x7f1a300c5c10 2023-11-28T11:56:38.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.611+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a3004e620 con 0x7f1a300c5c10 2023-11-28T11:56:38.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.612+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a28003a20 con 0x7f1a300c5c10 2023-11-28T11:56:38.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.612+0000 7f1a1effd700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1a100682f0 0x7f1a1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.629 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.612+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1a28085fa0 con 0x7f1a300c5c10 2023-11-28T11:56:38.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.612+0000 7f1a2e59c700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1a100682f0 0x7f1a1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.613+0000 7f1a1effd700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f1a1006e760 0x7f1a10070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.613+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 --> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a10071240 con 0x7f1a1006e760 2023-11-28T11:56:38.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.613+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a280885f0 con 0x7f1a300c5c10 2023-11-28T11:56:38.630 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.613+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f1a1006e760 0x7f1a10070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.613+0000 7f1a2e59c700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1a100682f0 0x7f1a1006a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a240010e0 tx=0x7f1a24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.613+0000 7f1a2dd9b700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f1a1006e760 0x7f1a10070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.631 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.614+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== osd.6 v2:172.21.15.177:6800/103771 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1a10071240 con 0x7f1a1006e760 2023-11-28T11:56:38.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.628+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 --> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a30155ea0 con 0x7f1a1006e760 2023-11-28T11:56:38.632 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a1effd700 1 -- 172.21.15.144:0/2998498420 <== osd.6 v2:172.21.15.177:6800/103771 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1a30155ea0 con 0x7f1a1006e760 2023-11-28T11:56:38.632 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:38.633 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:38.633 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:38.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f1a1006e760 msgr2=0x7f1a10070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.634 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.177:6800/103771,v1:172.21.15.177:6801/103771] conn(0x7f1a1006e760 0x7f1a10070b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1a100682f0 msgr2=0x7f1a1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1a100682f0 0x7f1a1006a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a240010e0 tx=0x7f1a24009040 comp rx=0 tx=0).stop 2023-11-28T11:56:38.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a300c5c10 msgr2=0x7f1a300c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 --2- 172.21.15.144:0/2998498420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a300c5c10 0x7f1a300c6080 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1a30070b90 tx=0x7f1a280104a0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.635 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a2ed9d700 1 -- 172.21.15.144:0/2998498420 reap_dead start 2023-11-28T11:56:38.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 shutdown_connections 2023-11-28T11:56:38.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 >> 172.21.15.144:0/2998498420 conn(0x7f1a3006b380 msgr2=0x7f1a3006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 shutdown_connections 2023-11-28T11:56:38.636 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.629+0000 7f1a34d0d700 1 -- 172.21.15.144:0/2998498420 wait complete. 2023-11-28T11:56:38.642 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T11:56:38.762 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.762+0000 7f8fc3fcb700 1 Processor -- start 2023-11-28T11:56:38.763 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.762+0000 7f8fc3fcb700 1 -- start start 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc3fcb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc071040 0x7f8fbc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc3fcb700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8fbc06dee0 0x7f8fbc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc3fcb700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fbc06cdc0 0x7f8fbc06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc3fcb700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc071950 con 0x7f8fbc06cdc0 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc3fcb700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc15d060 con 0x7f8fbc06dee0 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc3fcb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc15d340 con 0x7f8fbc071040 2023-11-28T11:56:38.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fbc06cdc0 0x7f8fbc06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fbc06cdc0 0x7f8fbc06d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39758/0 (socket says 172.21.15.144:39758) 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 -- 172.21.15.144:0/2861301330 learned_addr learned my addr 172.21.15.144:0/2861301330 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc1566700 1 --2- 172.21.15.144:0/2861301330 >> v2:172.21.15.177:3300/0 conn(0x7f8fbc06dee0 0x7f8fbc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 -- 172.21.15.144:0/2861301330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc071040 msgr2=0x7f8fbc071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc1d67700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc071040 0x7f8fbc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc071040 0x7f8fbc071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 -- 172.21.15.144:0/2861301330 >> v2:172.21.15.177:3300/0 conn(0x7f8fbc06dee0 msgr2=0x7f8fbc06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 --2- 172.21.15.144:0/2861301330 >> v2:172.21.15.177:3300/0 conn(0x7f8fbc06dee0 0x7f8fbc06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 -- 172.21.15.144:0/2861301330 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc15d620 con 0x7f8fbc06cdc0 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc1566700 1 --2- 172.21.15.144:0/2861301330 >> v2:172.21.15.177:3300/0 conn(0x7f8fbc06dee0 0x7f8fbc06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc1d67700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc071040 0x7f8fbc071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.763+0000 7f8fc2568700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06cdc0 0x7f8fbc06d240 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc09e140 tx=0x7f8fb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3ca446059917f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc0d65700 1 -- 172.21.15.144:0/2861301330 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400e070 con 0x7f8fbc06cdc0 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc0d65700 1 -- 172.21.15.144:0/2861301330 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb400b9c0 con 0x7f8fbc06cdc0 2023-11-28T11:56:38.771 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc0d65700 1 -- 172.21.15.144:0/2861301330 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400bd70 con 0x7f8fbc06cdc0 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/2861301330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06cdc0 msgr2=0x7f8fbc06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06cdc0 0x7f8fbc06d240 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc09e140 tx=0x7f8fb400b330 comp rx=0 tx=0).stop 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/2861301330 shutdown_connections 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06cdc0 0x7f8fbc06d240 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/2861301330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc071040 0x7f8fbc071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/2861301330 >> v2:172.21.15.177:3300/0 conn(0x7f8fbc06dee0 0x7f8fbc06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/2861301330 >> 172.21.15.144:0/2861301330 conn(0x7f8fbc0d0000 msgr2=0x7f8fbc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/2861301330 shutdown_connections 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.764+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/2861301330 wait complete. 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.765+0000 7f8fc3fcb700 1 Processor -- start 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.765+0000 7f8fc3fcb700 1 -- start start 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc3fcb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc06cdc0 0x7f8fbc1486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc3fcb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 0x7f8fbc158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.773 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc3fcb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fbc071040 0x7f8fbc1f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc3fcb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c50e0 con 0x7f8fbc06dee0 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc3fcb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c4af0 con 0x7f8fbc071040 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc3fcb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc148c10 con 0x7f8fbc06cdc0 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc1566700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 0x7f8fbc158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc1566700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 0x7f8fbc158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39762/0 (socket says 172.21.15.144:39762) 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc1566700 1 -- 172.21.15.144:0/3856180279 learned_addr learned my addr 172.21.15.144:0/3856180279 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:38.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc1d67700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc06cdc0 0x7f8fbc1486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.766+0000 7f8fc2568700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fbc071040 0x7f8fbc1f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc1566700 1 -- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc06cdc0 msgr2=0x7f8fbc1486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc1566700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc06cdc0 0x7f8fbc1486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc1566700 1 -- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fbc071040 msgr2=0x7f8fbc1f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc1566700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fbc071040 0x7f8fbc1f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc1566700 1 -- 172.21.15.144:0/3856180279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4009b30 con 0x7f8fbc06dee0 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc1566700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 0x7f8fbc158cb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c7f0 tx=0x7f8fac00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac011800 con 0x7f8fbc06dee0 2023-11-28T11:56:38.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac011960 con 0x7f8fbc06dee0 2023-11-28T11:56:38.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc06e890 con 0x7f8fbc06dee0 2023-11-28T11:56:38.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac011d10 con 0x7f8fbc06dee0 2023-11-28T11:56:38.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.767+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc15d900 con 0x7f8fbc06dee0 2023-11-28T11:56:38.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.768+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc15ccd0 con 0x7f8fbc06dee0 2023-11-28T11:56:38.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.768+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8fac021070 con 0x7f8fbc06dee0 2023-11-28T11:56:38.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.769+0000 7f8fb27fc700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8fa00682f0 0x7f8fa006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:38.777 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.769+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8fac0856b0 con 0x7f8fbc06dee0 2023-11-28T11:56:38.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.769+0000 7f8fc1d67700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8fa00682f0 0x7f8fa006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:38.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.769+0000 7f8fc1d67700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8fa00682f0 0x7f8fa006a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc158bf0 tx=0x7f8fb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:38.778 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.773+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8fac052180 con 0x7f8fbc06dee0 2023-11-28T11:56:38.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.956+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fbc0c5220 con 0x7f8fa00682f0 2023-11-28T11:56:38.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.956+0000 7f8fb27fc700 1 -- 172.21.15.144:0/3856180279 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8fbc0c5220 con 0x7f8fa00682f0 2023-11-28T11:56:38.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8fa00682f0 msgr2=0x7f8fa006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8fa00682f0 0x7f8fa006a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc158bf0 tx=0x7f8fb8008040 comp rx=0 tx=0).stop 2023-11-28T11:56:38.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 msgr2=0x7f8fbc158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:38.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 0x7f8fbc158cb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c7f0 tx=0x7f8fac00cb00 comp rx=0 tx=0).stop 2023-11-28T11:56:38.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 shutdown_connections 2023-11-28T11:56:38.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f8fa00682f0 0x7f8fa006a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fbc071040 0x7f8fbc1f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8fbc06cdc0 0x7f8fbc1486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 --2- 172.21.15.144:0/3856180279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fbc06dee0 0x7f8fbc158cb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:38.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 >> 172.21.15.144:0/3856180279 conn(0x7f8fbc0d0000 msgr2=0x7f8fbc06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:38.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 shutdown_connections 2023-11-28T11:56:38.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:38.958+0000 7f8fc3fcb700 1 -- 172.21.15.144:0/3856180279 wait complete. 2023-11-28T11:56:38.961 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T11:56:38.971 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T11:56:38.971 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:56:39.096 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.095+0000 7f1d71867700 1 Processor -- start 2023-11-28T11:56:39.096 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.095+0000 7f1d71867700 1 -- start start 2023-11-28T11:56:39.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d71867700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1d6c15aba0 0x7f1d6c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d71867700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c0b6ae0 0x7f1d6c0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d71867700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d6c0cfa20 0x7f1d6c0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.099 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d71867700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c15af70 con 0x7f1d6c15aba0 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d71867700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c0cc870 con 0x7f1d6c0cfa20 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d71867700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0cfec0 con 0x7f1d6c0b6ae0 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d6affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1d6c15aba0 0x7f1d6c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d6affd700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1d6c15aba0 0x7f1d6c0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39768/0 (socket says 172.21.15.144:39768) 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.096+0000 7f1d6affd700 1 -- 172.21.15.144:0/4200589753 learned_addr learned my addr 172.21.15.144:0/4200589753 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6b7fe700 1 --2- 172.21.15.144:0/4200589753 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c0cfa20 0x7f1d6c0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6affd700 1 -- 172.21.15.144:0/4200589753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c0b6ae0 msgr2=0x7f1d6c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:39.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6a7fc700 1 --2- 172.21.15.144:0/4200589753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c0b6ae0 0x7f1d6c0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6affd700 1 --2- 172.21.15.144:0/4200589753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c0b6ae0 0x7f1d6c0cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.101 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6affd700 1 -- 172.21.15.144:0/4200589753 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c0cfa20 msgr2=0x7f1d6c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6affd700 1 --2- 172.21.15.144:0/4200589753 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c0cfa20 0x7f1d6c0d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6affd700 1 -- 172.21.15.144:0/4200589753 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c0b4290 con 0x7f1d6c15aba0 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d6affd700 1 --2- 172.21.15.144:0/4200589753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15aba0 0x7f1d6c0b3d50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0ce7f0 tx=0x7f1d60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b8d2e7235ae076 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d69ffb700 1 -- 172.21.15.144:0/4200589753 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000b870 con 0x7f1d6c15aba0 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d69ffb700 1 -- 172.21.15.144:0/4200589753 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6000b9d0 con 0x7f1d6c15aba0 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d69ffb700 1 -- 172.21.15.144:0/4200589753 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000bd60 con 0x7f1d6c15aba0 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 -- 172.21.15.144:0/4200589753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15aba0 msgr2=0x7f1d6c0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.102 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 --2- 172.21.15.144:0/4200589753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15aba0 0x7f1d6c0b3d50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0ce7f0 tx=0x7f1d60009bd0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 -- 172.21.15.144:0/4200589753 shutdown_connections 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 --2- 172.21.15.144:0/4200589753 >> v2:172.21.15.177:3300/0 conn(0x7f1d6c0cfa20 0x7f1d6c0d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 --2- 172.21.15.144:0/4200589753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15aba0 0x7f1d6c0b3d50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 --2- 172.21.15.144:0/4200589753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c0b6ae0 0x7f1d6c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.097+0000 7f1d71867700 1 -- 172.21.15.144:0/4200589753 >> 172.21.15.144:0/4200589753 conn(0x7f1d6c06df70 msgr2=0x7f1d6c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.098+0000 7f1d71867700 1 -- 172.21.15.144:0/4200589753 shutdown_connections 2023-11-28T11:56:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.098+0000 7f1d71867700 1 -- 172.21.15.144:0/4200589753 wait complete. 2023-11-28T11:56:39.104 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.099+0000 7f1d71867700 1 Processor -- start 2023-11-28T11:56:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.099+0000 7f1d71867700 1 -- start start 2023-11-28T11:56:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c0cfa20 0x7f1d6c1f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c15aba0 0x7f1d6c14a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15ca80 0x7f1d6c06ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15d280 con 0x7f1d6c15ca80 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15d560 con 0x7f1d6c0cfa20 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c15d840 con 0x7f1d6c15aba0 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15ca80 0x7f1d6c06ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15ca80 0x7f1d6c06ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39778/0 (socket says 172.21.15.144:39778) 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 -- 172.21.15.144:0/887042821 learned_addr learned my addr 172.21.15.144:0/887042821 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:39.108 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 -- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c15aba0 msgr2=0x7f1d6c14a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d6c15aba0 0x7f1d6c14a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 -- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c0cfa20 msgr2=0x7f1d6c1f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d6c0cfa20 0x7f1d6c1f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 -- 172.21.15.144:0/887042821 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009550 con 0x7f1d6c15ca80 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d6b7fe700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15ca80 0x7f1d6c06ff60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400cd40 tx=0x7f1d6400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64003a20 con 0x7f1d6c15ca80 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d64003b80 con 0x7f1d6c15ca80 2023-11-28T11:56:39.109 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c0ccd80 con 0x7f1d6c15ca80 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004020 con 0x7f1d6c15ca80 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.100+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c0ccaa0 con 0x7f1d6c15ca80 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.101+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d6c0cd060 con 0x7f1d6c15ca80 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1d6401c020 con 0x7f1d6c15ca80 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d53fff700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d580682f0 0x7f1d5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1d64012070 con 0x7f1d6c15ca80 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d6affd700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d580682f0 0x7f1d5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.110 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d53fff700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7f1d5806e760 0x7f1d58070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 --> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d58071240 con 0x7f1d5806e760 2023-11-28T11:56:39.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d64088600 con 0x7f1d6c15ca80 2023-11-28T11:56:39.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.102+0000 7f1d6a7fc700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7f1d5806e760 0x7f1d58070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.103+0000 7f1d6affd700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d580682f0 0x7f1d5806a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009c10 tx=0x7f1d60010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.103+0000 7f1d6a7fc700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7f1d5806e760 0x7f1d58070b70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.111 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.104+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== osd.7 v2:172.21.15.177:6816/103773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1d58071240 con 0x7f1d5806e760 2023-11-28T11:56:39.118 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 --> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1d6c068690 con 0x7f1d5806e760 2023-11-28T11:56:39.119 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d53fff700 1 -- 172.21.15.144:0/887042821 <== osd.7 v2:172.21.15.177:6816/103773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1d6c068690 con 0x7f1d5806e760 2023-11-28T11:56:39.119 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2023-11-28T11:56:39.119 INFO:teuthology.orchestra.run.smithi144.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:56:39.119 INFO:teuthology.orchestra.run.smithi144.stdout:} 2023-11-28T11:56:39.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7f1d5806e760 msgr2=0x7f1d58070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.120 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.177:6816/103773,v1:172.21.15.177:6818/103773] conn(0x7f1d5806e760 0x7f1d58070b70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d580682f0 msgr2=0x7f1d5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f1d580682f0 0x7f1d5806a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009c10 tx=0x7f1d60010000 comp rx=0 tx=0).stop 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15ca80 msgr2=0x7f1d6c06ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 --2- 172.21.15.144:0/887042821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d6c15ca80 0x7f1d6c06ff60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400cd40 tx=0x7f1d6400a570 comp rx=0 tx=0).stop 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d6b7fe700 1 -- 172.21.15.144:0/887042821 reap_dead start 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 shutdown_connections 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.118+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 >> 172.21.15.144:0/887042821 conn(0x7f1d6c06df70 msgr2=0x7f1d6c159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:39.121 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.119+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 shutdown_connections 2023-11-28T11:56:39.122 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.119+0000 7f1d71867700 1 -- 172.21.15.144:0/887042821 wait complete. 2023-11-28T11:56:39.132 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T11:56:39.252 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.252+0000 7f1959a67700 1 Processor -- start 2023-11-28T11:56:39.253 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.252+0000 7f1959a67700 1 -- start start 2023-11-28T11:56:39.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1959a67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1954154e70 0x7f19540cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1959a67700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1954071750 0x7f195415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.255 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1959a67700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1954155240 con 0x7f19540cf360 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1959a67700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195415b100 con 0x7f1954071750 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1959a67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195415b3e0 con 0x7f1954154e70 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1953fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1954071750 0x7f195415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1952ffd700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f19540cf360 0x7f1954071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56498/0 (socket says 172.21.15.144:56498) 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1952ffd700 1 -- 172.21.15.144:0/2007617880 learned_addr learned my addr 172.21.15.144:0/2007617880 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f19537fe700 1 --2- 172.21.15.144:0/2007617880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1954154e70 0x7f19540cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1953fff700 1 -- 172.21.15.144:0/2007617880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1954154e70 msgr2=0x7f19540cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:39.256 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1953fff700 1 --2- 172.21.15.144:0/2007617880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1954154e70 0x7f19540cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1953fff700 1 -- 172.21.15.144:0/2007617880 >> v1:172.21.15.144:6789/0 conn(0x7f19540cf360 legacy=0x7f1954071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1953fff700 1 -- 172.21.15.144:0/2007617880 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195415b6c0 con 0x7f1954071750 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f19527fc700 1 -- 172.21.15.144:0/2007617880 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644859382 0 0) 0x7f1954155240 con 0x7f19540cf360 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f1953fff700 1 --2- 172.21.15.144:0/2007617880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954071750 0x7f195415e680 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f194c009d90 tx=0x7f194c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a35e459a77079be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f19527fc700 1 -- 172.21.15.144:0/2007617880 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00e070 con 0x7f1954071750 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f19527fc700 1 -- 172.21.15.144:0/2007617880 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00bbd0 con 0x7f1954071750 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.253+0000 7f19527fc700 1 -- 172.21.15.144:0/2007617880 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c012430 con 0x7f1954071750 2023-11-28T11:56:39.257 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 -- 172.21.15.144:0/2007617880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954071750 msgr2=0x7f195415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 --2- 172.21.15.144:0/2007617880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954071750 0x7f195415e680 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f194c009d90 tx=0x7f194c00b330 comp rx=0 tx=0).stop 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 -- 172.21.15.144:0/2007617880 shutdown_connections 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 --2- 172.21.15.144:0/2007617880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954071750 0x7f195415e680 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 --2- 172.21.15.144:0/2007617880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1954154e70 0x7f19540cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 -- 172.21.15.144:0/2007617880 >> 172.21.15.144:0/2007617880 conn(0x7f19540d0000 msgr2=0x7f19540d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 -- 172.21.15.144:0/2007617880 shutdown_connections 2023-11-28T11:56:39.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.254+0000 7f1959a67700 1 -- 172.21.15.144:0/2007617880 wait complete. 2023-11-28T11:56:39.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.255+0000 7f1959a67700 1 Processor -- start 2023-11-28T11:56:39.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.255+0000 7f1959a67700 1 -- start start 2023-11-28T11:56:39.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1959a67700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1954071750 0x7f1954152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1959a67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19540cf360 0x7f195406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.261 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1959a67700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 0x7f19541e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1959a67700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195406ce60 con 0x7f1954154e70 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1959a67700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195406d140 con 0x7f1954071750 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1959a67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195406d420 con 0x7f19540cf360 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1953fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 0x7f19541e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1953fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 0x7f19541e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39794/0 (socket says 172.21.15.144:39794) 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1953fff700 1 -- 172.21.15.144:0/1563002439 learned_addr learned my addr 172.21.15.144:0/1563002439 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f19537fe700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1954071750 0x7f1954152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.256+0000 7f1952ffd700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19540cf360 0x7f195406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1953fff700 1 -- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19540cf360 msgr2=0x7f195406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1953fff700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19540cf360 0x7f195406e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1953fff700 1 -- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1954071750 msgr2=0x7f1954152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1953fff700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1954071750 0x7f1954152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1953fff700 1 -- 172.21.15.144:0/1563002439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009a70 con 0x7f1954154e70 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1953fff700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 0x7f19541e43c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f194c013000 tx=0x7f194c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c003ea0 con 0x7f1954154e70 2023-11-28T11:56:39.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c020070 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19540af520 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c004280 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.257+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195415b9a0 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.258+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195415adb0 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.258+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f194c02e070 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.259+0000 7f193bfff700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19400682f0 0x7f194006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.259+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f194c022070 con 0x7f1954154e70 2023-11-28T11:56:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.259+0000 7f19537fe700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19400682f0 0x7f194006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.259+0000 7f19537fe700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19400682f0 0x7f194006a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1948006fd0 tx=0x7f1948008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.263+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f194c0541e0 con 0x7f1954154e70 2023-11-28T11:56:39.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.441+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f195406dc20 con 0x7f19400682f0 2023-11-28T11:56:39.442 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.442+0000 7f193bfff700 1 -- 172.21.15.144:0/1563002439 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f195406dc20 con 0x7f19400682f0 2023-11-28T11:56:39.447 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19400682f0 msgr2=0x7f194006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19400682f0 0x7f194006a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1948006fd0 tx=0x7f1948008040 comp rx=0 tx=0).stop 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 msgr2=0x7f19541e43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 0x7f19541e43c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f194c013000 tx=0x7f194c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 shutdown_connections 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1954154e70 0x7f19541e43c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f19400682f0 0x7f194006a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.448 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1954071750 0x7f1954152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 --2- 172.21.15.144:0/1563002439 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19540cf360 0x7f195406e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 >> 172.21.15.144:0/1563002439 conn(0x7f19540d0000 msgr2=0x7f19540ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:39.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 shutdown_connections 2023-11-28T11:56:39.449 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.444+0000 7f1959a67700 1 -- 172.21.15.144:0/1563002439 wait complete. 2023-11-28T11:56:39.450 INFO:teuthology.orchestra.run.smithi144.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T11:56:39.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:56:39.577 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 Processor -- start 2023-11-28T11:56:39.578 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 -- start start 2023-11-28T11:56:39.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5afc160540 0x7f5afc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5afc071950 con 0x7f5afc06dee0 2023-11-28T11:56:39.580 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc15d060 con 0x7f5afc160540 2023-11-28T11:56:39.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.577+0000 7f5b02919700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc15d340 con 0x7f5afc071040 2023-11-28T11:56:39.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afb7fe700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5afc06dee0 0x7f5afc06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56510/0 (socket says 172.21.15.144:56510) 2023-11-28T11:56:39.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afb7fe700 1 -- 172.21.15.144:0/2342279708 learned_addr learned my addr 172.21.15.144:0/2342279708 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:39.581 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5b00eb6700 1 --2- 172.21.15.144:0/2342279708 >> v2:172.21.15.177:3300/0 conn(0x7f5afc160540 0x7f5afc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afbfff700 1 --2- 172.21.15.144:0/2342279708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.582 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afaffd700 1 -- 172.21.15.144:0/2342279708 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520775794 0 0) 0x7f5afc071950 con 0x7f5afc06dee0 2023-11-28T11:56:39.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afaffd700 1 -- 172.21.15.144:0/2342279708 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae40035b0 con 0x7f5afc06dee0 2023-11-28T11:56:39.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afbfff700 1 -- 172.21.15.144:0/2342279708 >> v2:172.21.15.177:3300/0 conn(0x7f5afc160540 msgr2=0x7f5afc1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:39.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afbfff700 1 --2- 172.21.15.144:0/2342279708 >> v2:172.21.15.177:3300/0 conn(0x7f5afc160540 0x7f5afc1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afbfff700 1 -- 172.21.15.144:0/2342279708 >> v1:172.21.15.144:6789/0 conn(0x7f5afc06dee0 legacy=0x7f5afc06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.583 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afbfff700 1 -- 172.21.15.144:0/2342279708 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc15d620 con 0x7f5afc071040 2023-11-28T11:56:39.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afbfff700 1 --2- 172.21.15.144:0/2342279708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc071410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009840 tx=0x7f5aec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6469a521160b4dc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afaffd700 1 -- 172.21.15.144:0/2342279708 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec011830 con 0x7f5afc071040 2023-11-28T11:56:39.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afaffd700 1 -- 172.21.15.144:0/2342279708 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec011990 con 0x7f5afc071040 2023-11-28T11:56:39.584 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.578+0000 7f5afaffd700 1 -- 172.21.15.144:0/2342279708 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec011d20 con 0x7f5afc071040 2023-11-28T11:56:39.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 -- 172.21.15.144:0/2342279708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 msgr2=0x7f5afc071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 --2- 172.21.15.144:0/2342279708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc071410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009840 tx=0x7f5aec009b50 comp rx=0 tx=0).stop 2023-11-28T11:56:39.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 -- 172.21.15.144:0/2342279708 shutdown_connections 2023-11-28T11:56:39.585 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 --2- 172.21.15.144:0/2342279708 >> v2:172.21.15.177:3300/0 conn(0x7f5afc160540 0x7f5afc1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 --2- 172.21.15.144:0/2342279708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc071410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 -- 172.21.15.144:0/2342279708 >> 172.21.15.144:0/2342279708 conn(0x7f5afc0d0000 msgr2=0x7f5afc0d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:39.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 -- 172.21.15.144:0/2342279708 shutdown_connections 2023-11-28T11:56:39.586 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.579+0000 7f5b02919700 1 -- 172.21.15.144:0/2342279708 wait complete. 2023-11-28T11:56:39.589 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.580+0000 7f5b02919700 1 Processor -- start 2023-11-28T11:56:39.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.580+0000 7f5b02919700 1 -- start start 2023-11-28T11:56:39.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b02919700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5afc06dee0 0x7f5afc0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b02919700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b02919700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 0x7f5afc0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b02919700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0b59d0 con 0x7f5afc160540 2023-11-28T11:56:39.590 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b02919700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0b5540 con 0x7f5afc06dee0 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b02919700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0b4930 con 0x7f5afc071040 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b00eb6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 0x7f5afc0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b00eb6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 0x7f5afc0b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39808/0 (socket says 172.21.15.144:39808) 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5b00eb6700 1 -- 172.21.15.144:0/4129649080 learned_addr learned my addr 172.21.15.144:0/4129649080 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5afb7fe700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.581+0000 7f5afbfff700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5afc06dee0 0x7f5afc0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b00eb6700 1 -- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 msgr2=0x7f5afc0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.591 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b00eb6700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b00eb6700 1 -- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5afc06dee0 msgr2=0x7f5afc0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b00eb6700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5afc06dee0 0x7f5afc0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.592 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b00eb6700 1 -- 172.21.15.144:0/4129649080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0094a0 con 0x7f5afc160540 2023-11-28T11:56:39.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5afbfff700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5afc06dee0 0x7f5afc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:56:39.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5afb7fe700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:56:39.593 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b00eb6700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 0x7f5afc0b0f90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5af400efc0 tx=0x7f5af400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af400cce0 con 0x7f5afc160540 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af400f040 con 0x7f5afc160540 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc15ccd0 con 0x7f5afc160540 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4014480 con 0x7f5afc160540 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.582+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc15d900 con 0x7f5afc160540 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.583+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc06a270 con 0x7f5afc160540 2023-11-28T11:56:39.594 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.583+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5af40145e0 con 0x7f5afc160540 2023-11-28T11:56:39.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.584+0000 7f5af8ff9700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5ae80682f0 0x7f5ae806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:39.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.584+0000 7f5afbfff700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5ae80682f0 0x7f5ae806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:39.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.584+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5af40867f0 con 0x7f5afc160540 2023-11-28T11:56:39.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.584+0000 7f5afbfff700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5ae80682f0 0x7f5ae806a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0090b0 tx=0x7f5aec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:39.595 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.588+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5af4053280 con 0x7f5afc160540 2023-11-28T11:56:39.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.769+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5afc0b5680 con 0x7f5ae80682f0 2023-11-28T11:56:39.772 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.771+0000 7f5af8ff9700 1 -- 172.21.15.144:0/4129649080 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73388 (secure 0 0 0) 0x7f5afc0b5680 con 0x7f5ae80682f0 2023-11-28T11:56:39.773 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:56:39.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5ae80682f0 msgr2=0x7f5ae806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.774 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5ae80682f0 0x7f5ae806a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0090b0 tx=0x7f5aec008030 comp rx=0 tx=0).stop 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 msgr2=0x7f5afc0b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 0x7f5afc0b0f90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5af400efc0 tx=0x7f5af400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 shutdown_connections 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc160540 0x7f5afc0b0f90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f5ae80682f0 0x7f5ae806a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5afc06dee0 0x7f5afc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 --2- 172.21.15.144:0/4129649080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5afc071040 0x7f5afc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:39.775 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.773+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 >> 172.21.15.144:0/4129649080 conn(0x7f5afc0d0000 msgr2=0x7f5afc0cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.774+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 shutdown_connections 2023-11-28T11:56:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:39.774+0000 7f5b02919700 1 -- 172.21.15.144:0/4129649080 wait complete. 2023-11-28T11:56:39.776 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:56:39.786 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2023-11-28T11:56:38.558660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84224,"kb_used_data":7296,"kb_used_omap":0,"kb_used_meta":76864,"kb_avail":754890496,"statfs":{"total":773094113280,"available":773007867904,"internally_reserved":0,"allocated":7471104,"data_stored":3950676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78708736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.465123"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.140702+0000","last_change":"2023-11-28T11:51:11.827867+0000","last_active":"2023-11-28T11:56:30.140702+0000","last_peered":"2023-11-28T11:56:30.140702+0000","last_clean":"2023-11-28T11:56:30.140702+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:56:30.140702+0000","last_undegraded":"2023-11-28T11:56:30.140702+0000","last_fullsized":"2023-11-28T11:56:30.140702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.099134+0000","last_change":"2023-11-28T11:51:11.449639+0000","last_active":"2023-11-28T11:56:30.099134+0000","last_peered":"2023-11-28T11:56:30.099134+0000","last_clean":"2023-11-28T11:56:30.099134+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:56:30.099134+0000","last_undegraded":"2023-11-28T11:56:30.099134+0000","last_fullsized":"2023-11-28T11:56:30.099134+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:20:15.459781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":49,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.140984+0000","last_change":"2023-11-28T11:51:11.452703+0000","last_active":"2023-11-28T11:56:30.140984+0000","last_peered":"2023-11-28T11:56:30.140984+0000","last_clean":"2023-11-28T11:56:30.140984+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:56:30.140984+0000","last_undegraded":"2023-11-28T11:56:30.140984+0000","last_fullsized":"2023-11-28T11:56:30.140984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:15:46.276204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":200,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.096818+0000","last_change":"2023-11-28T11:51:12.448364+0000","last_active":"2023-11-28T11:56:30.096818+0000","last_peered":"2023-11-28T11:56:30.096818+0000","last_clean":"2023-11-28T11:56:30.096818+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:56:30.096818+0000","last_undegraded":"2023-11-28T11:56:30.096818+0000","last_fullsized":"2023-11-28T11:56:30.096818+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:11.428933+0000","last_clean_scrub_stamp":"2023-11-28T11:51:11.428933+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:49.911886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.097871+0000","last_change":"2023-11-28T11:51:11.699226+0000","last_active":"2023-11-28T11:56:30.097871+0000","last_peered":"2023-11-28T11:56:30.097871+0000","last_clean":"2023-11-28T11:56:30.097871+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:56:30.097871+0000","last_undegraded":"2023-11-28T11:56:30.097871+0000","last_fullsized":"2023-11-28T11:56:30.097871+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.096843+0000","last_change":"2023-11-28T11:51:11.699388+0000","last_active":"2023-11-28T11:56:30.096843+0000","last_peered":"2023-11-28T11:56:30.096843+0000","last_clean":"2023-11-28T11:56:30.096843+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:56:30.096843+0000","last_undegraded":"2023-11-28T11:56:30.096843+0000","last_fullsized":"2023-11-28T11:56:30.096843+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.139996+0000","last_change":"2023-11-28T11:51:11.699281+0000","last_active":"2023-11-28T11:56:30.139996+0000","last_peered":"2023-11-28T11:56:30.139996+0000","last_clean":"2023-11-28T11:56:30.139996+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:56:30.139996+0000","last_undegraded":"2023-11-28T11:56:30.139996+0000","last_fullsized":"2023-11-28T11:56:30.139996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:24.477084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.099804+0000","last_change":"2023-11-28T11:51:11.699188+0000","last_active":"2023-11-28T11:56:30.099804+0000","last_peered":"2023-11-28T11:56:30.099804+0000","last_clean":"2023-11-28T11:56:30.099804+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:56:30.099804+0000","last_undegraded":"2023-11-28T11:56:30.099804+0000","last_fullsized":"2023-11-28T11:56:30.099804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:16:46.524437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T11:56:30.099196+0000","last_change":"2023-11-28T11:51:11.452385+0000","last_active":"2023-11-28T11:56:30.099196+0000","last_peered":"2023-11-28T11:56:30.099196+0000","last_clean":"2023-11-28T11:56:30.099196+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:56:30.099196+0000","last_undegraded":"2023-11-28T11:56:30.099196+0000","last_fullsized":"2023-11-28T11:56:30.099196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:51:10.425275+0000","last_clean_scrub_stamp":"2023-11-28T11:51:10.425275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:07.444955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542213,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11020,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360820,"statfs":{"total":96636764160,"available":96625479680,"internally_reserved":0,"allocated":1380352,"data_stored":936857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54000000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70799999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59399999999999997,"5min":0.751,"15min":0.751},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.59999999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.55600000000000005},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67000000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64400000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65800000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542212,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9544,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362296,"statfs":{"total":96636764160,"available":96626991104,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.71199999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.72499999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.81599999999999995,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.68400000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51500000000000001,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.56899999999999995},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55000000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81299999999999994,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.62}]}]},{"osd":1,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10764,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361076,"statfs":{"total":96636764160,"available":96625741824,"internally_reserved":0,"allocated":1380352,"data_stored":936857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32300000000000001,"15min":0.32800000000000001},"min":{"1min":0.224,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.43099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.623},{"interface":"front","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34899999999999998},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.32900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53200000000000003,"5min":0.622,"15min":0.622},"last":0.42099999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86199999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80300000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75600000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542213,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13828,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94358012,"statfs":{"total":96636764160,"available":96622604288,"internally_reserved":0,"allocated":782336,"data_stored":345503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.32300000000000001},"min":{"1min":0.23599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47699999999999998,"5min":0.754,"15min":0.754},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35899999999999999,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.27100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.498,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56299999999999994,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.32900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71399999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":0.87,"15min":0.87},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.628},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.46600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57899999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542213,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.45200000000000001,"15min":0.64400000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.499,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.28499999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47699999999999998,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.51100000000000001,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.34699999999999998},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.53100000000000003,"5min":0.622,"15min":0.622},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49199999999999999,"15min":0.65800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54800000000000004,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.32800000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69099999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.89300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":8.359,"15min":8.359},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68799999999999994,"15min":0.83999999999999997},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88300000000000001,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.69799999999999995,"15min":0.878},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86099999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.58499999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.90600000000000003},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.75},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81200000000000006,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.61599999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9928,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361912,"statfs":{"total":96636764160,"available":96626597888,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":1.762,"15min":1.762},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.245}]},{"osd":1,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.51900000000000002,"15min":0.91800000000000004},"min":{"1min":0.253,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52500000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55800000000000005,"5min":1.857,"15min":1.857},"last":0.26700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.50800000000000001,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.59399999999999997,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0249999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.96799999999999997,"5min":1.498,"15min":1.498},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93799999999999994,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.55900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61599999999999999,"15min":0.56999999999999995},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.48199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96499999999999997,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67000000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542213,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9864,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361976,"statfs":{"total":96636764160,"available":96626663424,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.248,"15min":0.248},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.749},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":2.137,"15min":2.137},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71599999999999997,"5min":0.751,"15min":0.751},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.45400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.48399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.745,"15min":0.745},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67700000000000005,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.34899999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.76300000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63800000000000001,"5min":2.286,"15min":2.286},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.748}]},{"osd":7,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.63300000000000001},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72399999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542213,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9668,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362172,"statfs":{"total":96636764160,"available":96626864128,"internally_reserved":0,"allocated":782336,"data_stored":345503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.52100000000000002},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"min":{"1min":0.48299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.499}]},{"osd":1,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.42499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.36499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.53200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.497,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.679,"15min":1.679},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.749,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:56:39.836 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.836 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.837 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.837 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T11:51:11.428933+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=11, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.837 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.837 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.837 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.838 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.838 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T11:51:10.425275+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=51, tm_sec=10, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=56, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:56:39.838 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T11:56:53.454 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:56:53.457 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:56:59.839 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:56:59.972 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.970+0000 7f810b166700 1 Processor -- start 2023-11-28T11:56:59.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.970+0000 7f810b166700 1 -- start start 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.971+0000 7f810b166700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f81040ce040 0x7f810415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.971+0000 7f810b166700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81040d0a00 0x7f810406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.971+0000 7f810b166700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810406b420 con 0x7f81040b2930 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.971+0000 7f810b166700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810406b700 con 0x7f81040d0a00 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.971+0000 7f810b166700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810406b9e0 con 0x7f81040ce040 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.971+0000 7f8103fff700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81040d0a00 0x7f810406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:59.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8109703700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f81040b2930 0x7f810406c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56528/0 (socket says 172.21.15.144:56528) 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8109703700 1 -- 172.21.15.144:0/1424814668 learned_addr learned my addr 172.21.15.144:0/1424814668 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8108f02700 1 --2- 172.21.15.144:0/1424814668 >> v2:172.21.15.177:3300/0 conn(0x7f81040ce040 0x7f810415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8103fff700 1 -- 172.21.15.144:0/1424814668 >> v2:172.21.15.177:3300/0 conn(0x7f81040ce040 msgr2=0x7f810415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8103fff700 1 --2- 172.21.15.144:0/1424814668 >> v2:172.21.15.177:3300/0 conn(0x7f81040ce040 0x7f810415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8103fff700 1 -- 172.21.15.144:0/1424814668 >> v1:172.21.15.144:6789/0 conn(0x7f81040b2930 legacy=0x7f810406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8103fff700 1 -- 172.21.15.144:0/1424814668 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810406bcc0 con 0x7f81040d0a00 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f81037fe700 1 -- 172.21.15.144:0/1424814668 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776736945 0 0) 0x7f810406b420 con 0x7f81040b2930 2023-11-28T11:56:59.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f8103fff700 1 --2- 172.21.15.144:0/1424814668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f810406f850 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f810406eab0 tx=0x7f80f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe7b00da8122b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:59.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f81037fe700 1 -- 172.21.15.144:0/1424814668 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0011830 con 0x7f81040d0a00 2023-11-28T11:56:59.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f81037fe700 1 -- 172.21.15.144:0/1424814668 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f0011990 con 0x7f81040d0a00 2023-11-28T11:56:59.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.972+0000 7f81037fe700 1 -- 172.21.15.144:0/1424814668 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0011d20 con 0x7f81040d0a00 2023-11-28T11:56:59.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 -- 172.21.15.144:0/1424814668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 msgr2=0x7f810406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:59.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 --2- 172.21.15.144:0/1424814668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f810406f850 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f810406eab0 tx=0x7f80f0009b20 comp rx=0 tx=0).stop 2023-11-28T11:56:59.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 -- 172.21.15.144:0/1424814668 shutdown_connections 2023-11-28T11:56:59.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 --2- 172.21.15.144:0/1424814668 >> v2:172.21.15.177:3300/0 conn(0x7f81040ce040 0x7f810415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:59.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 --2- 172.21.15.144:0/1424814668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f810406f850 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:59.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 -- 172.21.15.144:0/1424814668 >> 172.21.15.144:0/1424814668 conn(0x7f810406aed0 msgr2=0x7f810406df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:56:59.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 -- 172.21.15.144:0/1424814668 shutdown_connections 2023-11-28T11:56:59.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.973+0000 7f810b166700 1 -- 172.21.15.144:0/1424814668 wait complete. 2023-11-28T11:56:59.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.974+0000 7f810b166700 1 Processor -- start 2023-11-28T11:56:59.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.974+0000 7f810b166700 1 -- start start 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f810b166700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81040b2930 0x7f81041f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f810b166700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81040ce040 0x7f81040cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f810b166700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f81040d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f810b166700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81041f6220 con 0x7f81040d0a00 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f810b166700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81041f5c60 con 0x7f81040ce040 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f810b166700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81041f5f40 con 0x7f81040b2930 2023-11-28T11:56:59.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f81040d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f81040d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39822/0 (socket says 172.21.15.144:39822) 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 -- 172.21.15.144:0/640151238 learned_addr learned my addr 172.21.15.144:0/640151238 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8103fff700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81040ce040 0x7f81040cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8108f02700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81040b2930 0x7f81041f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 -- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81040b2930 msgr2=0x7f81041f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81040b2930 0x7f81041f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 -- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81040ce040 msgr2=0x7f81040cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:56:59.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81040ce040 0x7f81040cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.975+0000 7f8109703700 1 -- 172.21.15.144:0/640151238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00094a0 con 0x7f81040d0a00 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.976+0000 7f8109703700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f81040d01e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00cba0 tx=0x7f80fc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.976+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00aaa0 con 0x7f81040d0a00 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.976+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc00ac20 con 0x7f81040d0a00 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.976+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0070d0 con 0x7f81040d0a00 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.976+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810406fd90 con 0x7f81040d0a00 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.976+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810406bfa0 con 0x7f81040d0a00 2023-11-28T11:56:59.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.977+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810406ff20 con 0x7f81040d0a00 2023-11-28T11:56:59.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.977+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80fc010070 con 0x7f81040d0a00 2023-11-28T11:56:59.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.978+0000 7f81017fa700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80e80682f0 0x7f80e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:56:59.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.978+0000 7f8108f02700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80e80682f0 0x7f80e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:56:59.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.978+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f80fc0852e0 con 0x7f81040d0a00 2023-11-28T11:56:59.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.978+0000 7f8108f02700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80e80682f0 0x7f80e806a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009b20 tx=0x7f80f80091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:56:59.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:56:59.982+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80fc051e40 con 0x7f81040d0a00 2023-11-28T11:57:00.166 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.164+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 --> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8104068690 con 0x7f80e80682f0 2023-11-28T11:57:00.168 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.166+0000 7f81017fa700 1 -- 172.21.15.144:0/640151238 <== mgr.4104 v2:172.21.15.144:6832/103287 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73422 (secure 0 0 0) 0x7f8104068690 con 0x7f80e80682f0 2023-11-28T11:57:00.171 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:57:00.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80e80682f0 msgr2=0x7f80e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80e80682f0 0x7f80e806a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009b20 tx=0x7f80f80091c0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.173 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 msgr2=0x7f81040d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f81040d01e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00cba0 tx=0x7f80fc00ceb0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.174 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 shutdown_connections 2023-11-28T11:57:00.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81040d0a00 0x7f81040d01e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f80e80682f0 0x7f80e806a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.175 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81040b2930 0x7f81041f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 --2- 172.21.15.144:0/640151238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81040ce040 0x7f81040cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 >> 172.21.15.144:0/640151238 conn(0x7f810406aed0 msgr2=0x7f810415df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:57:00.176 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 shutdown_connections 2023-11-28T11:57:00.177 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.168+0000 7f810b166700 1 -- 172.21.15.144:0/640151238 wait complete. 2023-11-28T11:57:00.177 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2023-11-28T11:57:00.184 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-11-28T11:56:58.562719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84288,"kb_used_data":7296,"kb_used_omap":0,"kb_used_meta":76928,"kb_avail":754890432,"statfs":{"total":773094113280,"available":773007802368,"internally_reserved":0,"allocated":7471104,"data_stored":3950676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78774272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002382"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:38.616524+0000","last_change":"2023-11-28T11:56:38.616492+0000","last_active":"2023-11-28T11:56:38.616524+0000","last_peered":"2023-11-28T11:56:38.616524+0000","last_clean":"2023-11-28T11:56:38.616524+0000","last_became_active":"2023-11-28T11:51:11.827635+0000","last_became_peered":"2023-11-28T11:51:11.827635+0000","last_unstale":"2023-11-28T11:56:38.616524+0000","last_undegraded":"2023-11-28T11:56:38.616524+0000","last_fullsized":"2023-11-28T11:56:38.616524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:38.616349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:38.616349+0000","last_clean_scrub_stamp":"2023-11-28T11:56:38.616349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:57.364643+0000","scrub_duration":0.003513721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:37.765087+0000","last_change":"2023-11-28T11:56:37.765064+0000","last_active":"2023-11-28T11:56:37.765087+0000","last_peered":"2023-11-28T11:56:37.765087+0000","last_clean":"2023-11-28T11:56:37.765087+0000","last_became_active":"2023-11-28T11:51:11.449361+0000","last_became_peered":"2023-11-28T11:51:11.449361+0000","last_unstale":"2023-11-28T11:56:37.765087+0000","last_undegraded":"2023-11-28T11:56:37.765087+0000","last_fullsized":"2023-11-28T11:56:37.765087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:37.764976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:37.764976+0000","last_clean_scrub_stamp":"2023-11-28T11:56:37.764976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:34.440506+0000","scrub_duration":0.0038043230000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":64,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:36.953445+0000","last_change":"2023-11-28T11:56:36.953396+0000","last_active":"2023-11-28T11:56:36.953445+0000","last_peered":"2023-11-28T11:56:36.953445+0000","last_clean":"2023-11-28T11:56:36.953445+0000","last_became_active":"2023-11-28T11:51:11.452377+0000","last_became_peered":"2023-11-28T11:51:11.452377+0000","last_unstale":"2023-11-28T11:56:36.953445+0000","last_undegraded":"2023-11-28T11:56:36.953445+0000","last_fullsized":"2023-11-28T11:56:36.953445+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:36.953245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:36.953245+0000","last_clean_scrub_stamp":"2023-11-28T11:56:36.953245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.806954+0000","scrub_duration":0.0037210670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"65'119","reported_seq":217,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:46.554914+0000","last_change":"2023-11-28T11:56:46.554865+0000","last_active":"2023-11-28T11:56:46.554914+0000","last_peered":"2023-11-28T11:56:46.554914+0000","last_clean":"2023-11-28T11:56:46.554914+0000","last_became_active":"2023-11-28T11:51:12.447149+0000","last_became_peered":"2023-11-28T11:51:12.447149+0000","last_unstale":"2023-11-28T11:56:46.554914+0000","last_undegraded":"2023-11-28T11:56:46.554914+0000","last_fullsized":"2023-11-28T11:56:46.554914+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"65'119","last_scrub_stamp":"2023-11-28T11:56:46.554755+0000","last_deep_scrub":"65'119","last_deep_scrub_stamp":"2023-11-28T11:56:46.554755+0000","last_clean_scrub_stamp":"2023-11-28T11:56:46.554755+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:49:57.437562+0000","scrub_duration":0.007139767,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:45.537642+0000","last_change":"2023-11-28T11:56:45.537614+0000","last_active":"2023-11-28T11:56:45.537642+0000","last_peered":"2023-11-28T11:56:45.537642+0000","last_clean":"2023-11-28T11:56:45.537642+0000","last_became_active":"2023-11-28T11:51:11.698874+0000","last_became_peered":"2023-11-28T11:51:11.698874+0000","last_unstale":"2023-11-28T11:56:45.537642+0000","last_undegraded":"2023-11-28T11:56:45.537642+0000","last_fullsized":"2023-11-28T11:56:45.537642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:45.537493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:45.537493+0000","last_clean_scrub_stamp":"2023-11-28T11:56:45.537493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:53.566569+0000","scrub_duration":0.0039006269999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:41.476868+0000","last_change":"2023-11-28T11:56:41.476839+0000","last_active":"2023-11-28T11:56:41.476868+0000","last_peered":"2023-11-28T11:56:41.476868+0000","last_clean":"2023-11-28T11:56:41.476868+0000","last_became_active":"2023-11-28T11:51:11.698989+0000","last_became_peered":"2023-11-28T11:51:11.698989+0000","last_unstale":"2023-11-28T11:56:41.476868+0000","last_undegraded":"2023-11-28T11:56:41.476868+0000","last_fullsized":"2023-11-28T11:56:41.476868+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:41.476707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:41.476707+0000","last_clean_scrub_stamp":"2023-11-28T11:56:41.476707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:35:46.233188+0000","scrub_duration":0.0040382689999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":68,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:40.567150+0000","last_change":"2023-11-28T11:56:40.567113+0000","last_active":"2023-11-28T11:56:40.567150+0000","last_peered":"2023-11-28T11:56:40.567150+0000","last_clean":"2023-11-28T11:56:40.567150+0000","last_became_active":"2023-11-28T11:51:11.698872+0000","last_became_peered":"2023-11-28T11:51:11.698872+0000","last_unstale":"2023-11-28T11:56:40.567150+0000","last_undegraded":"2023-11-28T11:56:40.567150+0000","last_fullsized":"2023-11-28T11:56:40.567150+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:40.566988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:40.566988+0000","last_clean_scrub_stamp":"2023-11-28T11:56:40.566988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:07:49.978379+0000","scrub_duration":0.0039303089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:39.561627+0000","last_change":"2023-11-28T11:56:39.561578+0000","last_active":"2023-11-28T11:56:39.561627+0000","last_peered":"2023-11-28T11:56:39.561627+0000","last_clean":"2023-11-28T11:56:39.561627+0000","last_became_active":"2023-11-28T11:51:11.698870+0000","last_became_peered":"2023-11-28T11:51:11.698870+0000","last_unstale":"2023-11-28T11:56:39.561627+0000","last_undegraded":"2023-11-28T11:56:39.561627+0000","last_fullsized":"2023-11-28T11:56:39.561627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:39.561357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:39.561357+0000","last_clean_scrub_stamp":"2023-11-28T11:56:39.561357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:23.287037+0000","scrub_duration":0.0037794840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:56:42.602396+0000","last_change":"2023-11-28T11:56:42.602366+0000","last_active":"2023-11-28T11:56:42.602396+0000","last_peered":"2023-11-28T11:56:42.602396+0000","last_clean":"2023-11-28T11:56:42.602396+0000","last_became_active":"2023-11-28T11:51:11.452021+0000","last_became_peered":"2023-11-28T11:51:11.452021+0000","last_unstale":"2023-11-28T11:56:42.602396+0000","last_undegraded":"2023-11-28T11:56:42.602396+0000","last_fullsized":"2023-11-28T11:56:42.602396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:56:42.602252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:56:42.602252+0000","last_clean_scrub_stamp":"2023-11-28T11:56:42.602252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:01.350546+0000","scrub_duration":0.0040649420000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542217,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11020,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360820,"statfs":{"total":96636764160,"available":96625479680,"internally_reserved":0,"allocated":1380352,"data_stored":936857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.434,"5min":0.378,"15min":0.378},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.70799999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.51300000000000001},"min":{"1min":0.34000000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.59399999999999997,"5min":0.751,"15min":0.751},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.68899999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.55600000000000005},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.625,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.67000000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.64400000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66700000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.36199999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9544,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362296,"statfs":{"total":96636764160,"available":96626991104,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.753,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60099999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.42899999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.50700000000000001},"min":{"1min":0.38600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.497},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.77300000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.81599999999999995,"5min":3.7810000000000001,"15min":3.7810000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.496,"15min":0.52100000000000002},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51500000000000001,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.36099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52700000000000002,"15min":0.56899999999999995},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77800000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.58999999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71799999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.434}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55200000000000005,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80100000000000005,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81299999999999994,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.44700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10828,"kb_used_data":1348,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94361012,"statfs":{"total":96636764160,"available":96625676288,"internally_reserved":0,"allocated":1380352,"data_stored":936857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.32300000000000001,"15min":0.32800000000000001},"min":{"1min":0.224,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.43099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34899999999999998},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26100000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.50900000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.371},"min":{"1min":0.25700000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53200000000000003,"5min":0.622,"15min":0.622},"last":0.45400000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42399999999999999,"15min":0.41299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39600000000000002,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80800000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58499999999999996,"15min":0.56799999999999995},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73799999999999999,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86199999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77900000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63400000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:56:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.432,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77200000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58099999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542217,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13828,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":13056,"kb_avail":94358012,"statfs":{"total":96636764160,"available":96622604288,"internally_reserved":0,"allocated":782336,"data_stored":345503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":13369344},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.32300000000000001},"min":{"1min":0.23599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.47699999999999998,"5min":0.754,"15min":0.754},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35899999999999999,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.45600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.376,"15min":0.35599999999999998},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52200000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.432},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.35899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.504}]},{"osd":3,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.374,"15min":0.34399999999999997},"min":{"1min":0.26000000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.498,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.441},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.42399999999999999,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56299999999999994,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.52900000000000003},"min":{"1min":0.41999999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71399999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84099999999999997,"5min":0.87,"15min":0.87},"last":0.78200000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59499999999999997,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59799999999999998,"15min":0.54100000000000004},"min":{"1min":0.44500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.751,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.46600000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79800000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.58699999999999997},"min":{"1min":0.45900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83299999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78900000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81899999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542217,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9608,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362232,"statfs":{"total":96636764160,"available":96626925568,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35799999999999998,"15min":0.34999999999999998},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":0.50700000000000001,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.45200000000000001,"15min":0.64400000000000002},"min":{"1min":0.249,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.499,"5min":8.2970000000000006,"15min":8.2970000000000006},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38600000000000001,"15min":0.38500000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47699999999999998,"15min":0.66600000000000004},"min":{"1min":0.29399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.51100000000000001,"5min":8.4039999999999999,"15min":8.4039999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.34699999999999998},"min":{"1min":0.30399999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.53100000000000003,"5min":0.622,"15min":0.622},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.49199999999999999,"15min":0.65800000000000003},"min":{"1min":0.27300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54800000000000004,"5min":7.3440000000000003,"15min":7.3440000000000003},"last":0.54500000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.69099999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.89300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":8.359,"15min":8.359},"last":0.73699999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60499999999999998,"15min":0.57299999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83899999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.68799999999999994,"15min":0.83999999999999997},"min":{"1min":0.505,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88300000000000001,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.69799999999999995,"15min":0.878},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":8.3810000000000002,"15min":8.3810000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.63,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86099999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.97499999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.71199999999999997,"15min":0.90600000000000003},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":8.3339999999999996,"15min":8.3339999999999996},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.67000000000000004,"15min":0.75},"min":{"1min":0.433,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81200000000000006,"5min":4.6820000000000004,"15min":4.6820000000000004},"last":0.77700000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9928,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361912,"statfs":{"total":96636764160,"available":96626597888,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36599999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54000000000000004,"5min":1.762,"15min":1.762},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.35799999999999998},"min":{"1min":0.25700000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.51900000000000002,"15min":0.91800000000000004},"min":{"1min":0.253,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.52500000000000002,"5min":16.231000000000002,"15min":16.231000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.42099999999999999},"min":{"1min":0.27600000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.55800000000000005,"5min":1.857,"15min":1.857},"last":0.45200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.50800000000000001,"15min":0.68400000000000005},"min":{"1min":0.32700000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.59399999999999997,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.627,"15min":0.59699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0249999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.441,"5min":0.317,"15min":0.317},"max":{"1min":0.96799999999999997,"5min":1.498,"15min":1.498},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93799999999999994,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61599999999999999,"15min":0.56999999999999995},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.48199999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.96499999999999997,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95599999999999996,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63900000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542217,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9864,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361976,"statfs":{"total":96636764160,"available":96626663424,"internally_reserved":0,"allocated":786432,"data_stored":346489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.248,"15min":0.248},"max":{"1min":0.70699999999999996,"5min":0.749,"15min":0.749},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53000000000000003,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":2.137,"15min":2.137},"last":0.72099999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.50700000000000001},"min":{"1min":0.39700000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71599999999999997,"5min":0.751,"15min":0.751},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.50900000000000001},"min":{"1min":0.45400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.48399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.53800000000000003,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.745,"15min":0.745},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.67700000000000005,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.34899999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68200000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.751,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47099999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.52000000000000002,"15min":0.59399999999999997},"min":{"1min":0.34300000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63800000000000001,"5min":2.286,"15min":2.286},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.63300000000000001},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69299999999999995,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.621},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72299999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60599999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542217,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9668,"kb_used_data":764,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362172,"statfs":{"total":96636764160,"available":96626864128,"internally_reserved":0,"allocated":782336,"data_stored":345503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.52100000000000002},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"min":{"1min":0.48299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.749,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64000000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.501},"min":{"1min":0.42499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.68300000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76300000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56699999999999995,"15min":0.53200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.497,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.755,"5min":1.679,"15min":1.679},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57799999999999996,"15min":0.56699999999999995},"min":{"1min":0.42399999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57199999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73299999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.749,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.39900000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:56:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:57:00.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T11:57:00.317 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.309+0000 7f687e548700 1 Processor -- start 2023-11-28T11:57:00.317 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.309+0000 7f687e548700 1 -- start start 2023-11-28T11:57:00.317 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.309+0000 7f687e548700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f687815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.309+0000 7f687e548700 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687806b420 con 0x7f68780b2930 2023-11-28T11:57:00.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.309+0000 7f687e548700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687806b700 con 0x7f68780d0a00 2023-11-28T11:57:00.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.309+0000 7f687e548700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687806ff60 con 0x7f68780ce040 2023-11-28T11:57:00.318 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f687cae5700 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f68780b2930 0x7f687806c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:56542/0 (socket says 172.21.15.144:56542) 2023-11-28T11:57:00.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f687cae5700 1 -- 172.21.15.144:0/2895852302 learned_addr learned my addr 172.21.15.144:0/2895852302 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T11:57:00.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6877fff700 1 --2- 172.21.15.144:0/2895852302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f687815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6876ffd700 1 -- 172.21.15.144:0/2895852302 <== mon.0 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356202881 0 0) 0x7f687806b420 con 0x7f68780b2930 2023-11-28T11:57:00.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6876ffd700 1 -- 172.21.15.144:0/2895852302 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68600035b0 con 0x7f68780b2930 2023-11-28T11:57:00.319 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6877fff700 1 -- 172.21.15.144:0/2895852302 >> v1:172.21.15.177:6789/0 conn(0x7f68780d0a00 legacy=0x7f687806f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6876ffd700 1 -- 172.21.15.144:0/2895852302 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368597947 0 0) 0x7f687806b700 con 0x7f68780d0a00 2023-11-28T11:57:00.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6877fff700 1 -- 172.21.15.144:0/2895852302 >> v1:172.21.15.144:6789/0 conn(0x7f68780b2930 legacy=0x7f687806c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6877fff700 1 -- 172.21.15.144:0/2895852302 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687806b9e0 con 0x7f68780ce040 2023-11-28T11:57:00.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6877fff700 1 --2- 172.21.15.144:0/2895852302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f687815df10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6868009850 tx=0x7f6868009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9114aba184036e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:57:00.320 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6876ffd700 1 -- 172.21.15.144:0/2895852302 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68700019f0 con 0x7f68780ce040 2023-11-28T11:57:00.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6876ffd700 1 -- 172.21.15.144:0/2895852302 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868005c30 con 0x7f68780ce040 2023-11-28T11:57:00.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.310+0000 7f6876ffd700 1 -- 172.21.15.144:0/2895852302 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868006030 con 0x7f68780ce040 2023-11-28T11:57:00.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 -- 172.21.15.144:0/2895852302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 msgr2=0x7f687815df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 --2- 172.21.15.144:0/2895852302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f687815df10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6868009850 tx=0x7f6868009b60 comp rx=0 tx=0).stop 2023-11-28T11:57:00.321 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 -- 172.21.15.144:0/2895852302 shutdown_connections 2023-11-28T11:57:00.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 --2- 172.21.15.144:0/2895852302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f687815df10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 -- 172.21.15.144:0/2895852302 >> 172.21.15.144:0/2895852302 conn(0x7f687806aed0 msgr2=0x7f687806df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:57:00.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 -- 172.21.15.144:0/2895852302 shutdown_connections 2023-11-28T11:57:00.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.311+0000 7f687e548700 1 -- 172.21.15.144:0/2895852302 wait complete. 2023-11-28T11:57:00.322 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.312+0000 7f687e548700 1 Processor -- start 2023-11-28T11:57:00.323 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.312+0000 7f687e548700 1 -- start start 2023-11-28T11:57:00.323 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f687e548700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 0x7f6878159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.323 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f687e548700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f6878157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f687e548700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68780d0a00 0x7f68780c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.325 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f687e548700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878150e20 con 0x7f68780b2930 2023-11-28T11:57:00.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f687e548700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878159400 con 0x7f68780d0a00 2023-11-28T11:57:00.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f687e548700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6878158ae0 con 0x7f68780ce040 2023-11-28T11:57:00.326 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f6877fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 0x7f6878159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f6877fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 0x7f6878159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39832/0 (socket says 172.21.15.144:39832) 2023-11-28T11:57:00.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.313+0000 7f6877fff700 1 -- 172.21.15.144:0/3038952854 learned_addr learned my addr 172.21.15.144:0/3038952854 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:57:00.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f68777fe700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f6878157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f687cae5700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68780d0a00 0x7f68780c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.327 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6877fff700 1 -- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 msgr2=0x7f6878157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6877fff700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f6878157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6877fff700 1 -- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68780d0a00 msgr2=0x7f68780c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6877fff700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68780d0a00 0x7f68780c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6877fff700 1 -- 172.21.15.144:0/3038952854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868009430 con 0x7f68780b2930 2023-11-28T11:57:00.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6877fff700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 0x7f6878159720 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6868011000 tx=0x7f6868004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:57:00.328 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868012680 con 0x7f68780b2930 2023-11-28T11:57:00.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868012800 con 0x7f68780b2930 2023-11-28T11:57:00.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687806bfa0 con 0x7f68780b2930 2023-11-28T11:57:00.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868012bb0 con 0x7f68780b2930 2023-11-28T11:57:00.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.314+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687806bcc0 con 0x7f68780b2930 2023-11-28T11:57:00.329 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.315+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68780b2db0 con 0x7f68780b2930 2023-11-28T11:57:00.330 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.315+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f686802e070 con 0x7f68780b2930 2023-11-28T11:57:00.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.316+0000 7f6874ff9700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f68640682f0 0x7f686406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.316+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6868087150 con 0x7f68780b2930 2023-11-28T11:57:00.331 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.316+0000 7f68777fe700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f68640682f0 0x7f686406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.316+0000 7f68777fe700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f68640682f0 0x7f686406a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f68780d0700 tx=0x7f686c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:57:00.332 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.320+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6868053cb0 con 0x7f68780b2930 2023-11-28T11:57:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.504+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6878158240 con 0x7f68780b2930 2023-11-28T11:57:00.508 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.507+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6868053cb0 con 0x7f68780b2930 2023-11-28T11:57:00.509 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.507+0000 7f6874ff9700 1 -- 172.21.15.144:0/3038952854 <== mon.0 v2:172.21.15.144:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f68680572d0 con 0x7f68780b2930 2023-11-28T11:57:00.512 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.510+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f68640682f0 msgr2=0x7f686406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.512 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.510+0000 7f687e548700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f68640682f0 0x7f686406a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f68780d0700 tx=0x7f686c009040 comp rx=0 tx=0).stop 2023-11-28T11:57:00.512 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.510+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 msgr2=0x7f6878159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.510+0000 7f687e548700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 0x7f6878159720 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6868011000 tx=0x7f6868004750 comp rx=0 tx=0).stop 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.511+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 shutdown_connections 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.511+0000 7f687e548700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68780b2930 0x7f6878159720 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.511+0000 7f687e548700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f68640682f0 0x7f686406a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.511+0000 7f687e548700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68780d0a00 0x7f68780c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.511+0000 7f687e548700 1 --2- 172.21.15.144:0/3038952854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68780ce040 0x7f6878157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.513 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.511+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 >> 172.21.15.144:0/3038952854 conn(0x7f687806aed0 msgr2=0x7f687815f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:57:00.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.512+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 shutdown_connections 2023-11-28T11:57:00.514 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.512+0000 7f687e548700 1 -- 172.21.15.144:0/3038952854 wait complete. 2023-11-28T11:57:00.525 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:57:00.525 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:57:00.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.645+0000 7f9f6bdf1700 1 Processor -- start 2023-11-28T11:57:00.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.645+0000 7f9f6bdf1700 1 -- start start 2023-11-28T11:57:00.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6bdf1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f64071040 0x7f9f64071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6bdf1700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f6406dee0 0x7f9f6406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6bdf1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406cdc0 0x7f9f6406d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6bdf1700 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f64071950 con 0x7f9f6406dee0 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6bdf1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6415d060 con 0x7f9f64071040 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6bdf1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6415d340 con 0x7f9f6406cdc0 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f6406dee0 0x7f9f6406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f6406dee0 0x7f9f6406e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39840/0 (socket says 172.21.15.144:39840) 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 -- 172.21.15.144:0/2982926624 learned_addr learned my addr 172.21.15.144:0/2982926624 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f69b8d700 1 --2- 172.21.15.144:0/2982926624 >> v2:172.21.15.177:3300/0 conn(0x7f9f64071040 0x7f9f64071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6a38e700 1 --2- 172.21.15.144:0/2982926624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406cdc0 0x7f9f6406d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 -- 172.21.15.144:0/2982926624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406cdc0 msgr2=0x7f9f6406d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 --2- 172.21.15.144:0/2982926624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406cdc0 0x7f9f6406d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 -- 172.21.15.144:0/2982926624 >> v2:172.21.15.177:3300/0 conn(0x7f9f64071040 msgr2=0x7f9f64071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 --2- 172.21.15.144:0/2982926624 >> v2:172.21.15.177:3300/0 conn(0x7f9f64071040 0x7f9f64071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 -- 172.21.15.144:0/2982926624 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6415d620 con 0x7f9f6406dee0 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f6938c700 1 --2- 172.21.15.144:0/2982926624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6406dee0 0x7f9f6406e330 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9f640d0f50 tx=0x7f9f54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37885e88d5f73fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f68b8b700 1 -- 172.21.15.144:0/2982926624 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400b870 con 0x7f9f6406dee0 2023-11-28T11:57:00.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f68b8b700 1 -- 172.21.15.144:0/2982926624 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9f5400b9d0 con 0x7f9f6406dee0 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.646+0000 7f9f68b8b700 1 -- 172.21.15.144:0/2982926624 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400bd80 con 0x7f9f6406dee0 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/2982926624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6406dee0 msgr2=0x7f9f6406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/2982926624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6406dee0 0x7f9f6406e330 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9f640d0f50 tx=0x7f9f54009bd0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/2982926624 shutdown_connections 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/2982926624 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406cdc0 0x7f9f6406d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/2982926624 >> v2:172.21.15.177:3300/0 conn(0x7f9f64071040 0x7f9f64071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/2982926624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6406dee0 0x7f9f6406e330 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/2982926624 >> 172.21.15.144:0/2982926624 conn(0x7f9f640d0000 msgr2=0x7f9f640d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:57:00.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/2982926624 shutdown_connections 2023-11-28T11:57:00.653 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.647+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/2982926624 wait complete. 2023-11-28T11:57:00.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.648+0000 7f9f6bdf1700 1 Processor -- start 2023-11-28T11:57:00.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.648+0000 7f9f6bdf1700 1 -- start start 2023-11-28T11:57:00.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6bdf1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6406cdc0 0x7f9f640b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6bdf1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406dee0 0x7f9f640b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6bdf1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 0x7f9f640b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6bdf1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b3a90 con 0x7f9f64071040 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6bdf1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641f6860 con 0x7f9f6406cdc0 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6bdf1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640b5d20 con 0x7f9f6406dee0 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 0x7f9f640b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 0x7f9f640b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39850/0 (socket says 172.21.15.144:39850) 2023-11-28T11:57:00.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 -- 172.21.15.144:0/645215955 learned_addr learned my addr 172.21.15.144:0/645215955 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f69b8d700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6406cdc0 0x7f9f640b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6938c700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406dee0 0x7f9f640b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 -- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406dee0 msgr2=0x7f9f640b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406dee0 0x7f9f640b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 -- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6406cdc0 msgr2=0x7f9f640b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6406cdc0 0x7f9f640b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.657 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.649+0000 7f9f6a38e700 1 -- 172.21.15.144:0/645215955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54009550 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.650+0000 7f9f6a38e700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 0x7f9f640b4fd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c007d20 tx=0x7f9f5c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.650+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00ed30 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.650+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9f5c007e70 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.650+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c014660 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.650+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6406e870 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.650+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6415d900 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.651+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6415ccd0 con 0x7f9f64071040 2023-11-28T11:57:00.658 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.651+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f5c012050 con 0x7f9f64071040 2023-11-28T11:57:00.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.651+0000 7f9f5a7fc700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9f480682f0 0x7f9f4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:57:00.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.652+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9f5c085df0 con 0x7f9f64071040 2023-11-28T11:57:00.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.652+0000 7f9f69b8d700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9f480682f0 0x7f9f4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:57:00.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.652+0000 7f9f69b8d700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9f480682f0 0x7f9f4806a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f640c6aa0 tx=0x7f9f60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:57:00.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.656+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f5c052950 con 0x7f9f64071040 2023-11-28T11:57:00.880 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.879+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f640c3ce0 con 0x7f9f64071040 2023-11-28T11:57:00.881 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.879+0000 7f9f5a7fc700 1 -- 172.21.15.144:0/645215955 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f9f5c055f70 con 0x7f9f64071040 2023-11-28T11:57:00.881 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T11:57:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/418551146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":418551146},{"type":"v1","addr":"172.21.15.177:6835","nonce":418551146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5570,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/740385805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":740385805},{"type":"v1","addr":"172.21.15.144:6837","nonce":740385805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5611,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/100594470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":100594470},{"type":"v1","addr":"172.21.15.177:6834","nonce":100594470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5985,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1085068845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1085068845},{"type":"v1","addr":"172.21.15.144:6836","nonce":1085068845}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-11-28T11:57:00.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9f480682f0 msgr2=0x7f9f4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9f480682f0 0x7f9f4806a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f640c6aa0 tx=0x7f9f60008040 comp rx=0 tx=0).stop 2023-11-28T11:57:00.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 msgr2=0x7f9f640b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:57:00.883 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 0x7f9f640b4fd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c007d20 tx=0x7f9f5c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 shutdown_connections 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f64071040 0x7f9f640b4fd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.144:6832/103287,v1:172.21.15.144:6833/103287] conn(0x7f9f480682f0 0x7f9f4806a7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f6406cdc0 0x7f9f640b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 --2- 172.21.15.144:0/645215955 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f6406dee0 0x7f9f640b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 >> 172.21.15.144:0/645215955 conn(0x7f9f640d0000 msgr2=0x7f9f640cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:57:00.884 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 shutdown_connections 2023-11-28T11:57:00.885 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T11:57:00.881+0000 7f9f6bdf1700 1 -- 172.21.15.144:0/645215955 wait complete. 2023-11-28T11:57:00.886 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2023-11-28T11:57:00.895 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T11:57:00.895 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T11:57:00.896 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:57:00.896 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:00.904 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:57:00.904 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:57:00.904 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:00.953 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:57:00.954 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:57:00.954 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:00.964 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:57:00.965 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:57:00.965 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.016 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:57:01.016 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T11:57:01.017 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T11:57:01.017 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.047 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T11:57:01.048 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T11:57:01.048 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.114 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T11:57:01.114 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T11:57:01.114 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.188 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T11:57:01.189 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T11:57:01.189 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.261 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T11:57:01.262 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T11:57:01.262 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.299 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T11:57:01.300 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T11:57:01.300 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.374 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T11:57:01.375 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T11:57:01.376 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.453 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T11:57:01.454 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T11:57:01.454 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.534 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T11:57:01.534 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T11:57:01.534 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T11:57:01.534 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.559 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T11:57:01.559 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T11:57:01.559 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.599 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T11:57:01.599 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T11:57:01.599 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T11:57:01.600 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.610 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T11:57:01.610 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T11:57:01.611 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.650 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T11:57:01.651 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T11:57:01.651 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:57:01.703 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T11:57:01.703 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T11:57:01.704 DEBUG:teuthology.orchestra.run.smithi144:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T11:57:01.736 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:57:01.737 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T11:57:01.987 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:57:01.988 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T11:57:02.081 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:57:02.082 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T11:57:02.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:57:02.146 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T11:57:02.230 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:57:02.231 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T11:57:02.472 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:57:02.472 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T11:57:02.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:57:02.585 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T11:57:02.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:57:02.652 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T11:57:02.746 INFO:tasks.ceph:Archiving mon data... 2023-11-28T11:57:02.747 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/data/mon.a.tgz 2023-11-28T11:57:02.747 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-11-28T11:57:02.766 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.WXFazzxl4v 2023-11-28T11:57:02.766 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WXFazzxl4v 2023-11-28T11:57:02.918 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.WXFazzxl4v 2023-11-28T11:57:03.005 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.WXFazzxl4v is 402KB 2023-11-28T11:57:03.101 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.WXFazzxl4v 2023-11-28T11:57:03.117 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/data/mon.b.tgz 2023-11-28T11:57:03.118 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-11-28T11:57:03.135 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.1LGnIeqr4R 2023-11-28T11:57:03.136 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1LGnIeqr4R 2023-11-28T11:57:03.309 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.1LGnIeqr4R 2023-11-28T11:57:03.395 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.1LGnIeqr4R is 526KB 2023-11-28T11:57:03.496 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.1LGnIeqr4R 2023-11-28T11:57:03.512 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/data/mon.c.tgz 2023-11-28T11:57:03.512 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-11-28T11:57:03.570 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.oERmpb0rHL 2023-11-28T11:57:03.570 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.oERmpb0rHL 2023-11-28T11:57:03.736 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.oERmpb0rHL 2023-11-28T11:57:03.823 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.oERmpb0rHL is 527KB 2023-11-28T11:57:03.921 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.oERmpb0rHL 2023-11-28T11:57:03.937 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T11:57:03.938 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:57:03.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:57:04.004 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T11:57:04.004 DEBUG:teuthology.orchestra.run.smithi144:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:57:04.012 DEBUG:teuthology.orchestra.run.smithi177:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:57:04.057 INFO:teuthology.orchestra.run.smithi144.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:57:04.075 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:57:04.081 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T11:57:04.083 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/remote/smithi144/crash 2023-11-28T11:57:04.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:57:04.113 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/remote/smithi177/crash 2023-11-28T11:57:04.114 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:57:04.146 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T11:57:04.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:57:04.157 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:57:04.234 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:57:04.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:57:04.288 INFO:tasks.ceph:Compressing logs... 2023-11-28T11:57:04.288 DEBUG:teuthology.orchestra.run.smithi144:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:57:04.305 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:57:06.039 INFO:tasks.ceph:Archiving logs... 2023-11-28T11:57:06.039 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/remote/smithi144/log 2023-11-28T11:57:06.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:57:06.583 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/remote/smithi177/log 2023-11-28T11:57:06.584 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:57:06.825 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T11:57:06.836 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T11:57:06.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:57:06.839 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:57:06.927 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T11:57:06.937 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T11:57:06.937 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:57:06.939 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:57:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:bash: ntpq: command not found 2023-11-28T11:57:06.956 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-11-28T11:57:06.957 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:57:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2023-11-28T11:57:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +832us[ +832us] +/- 82ms 2023-11-28T11:57:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 -642us[ -642us] +/- 49ms 2023-11-28T11:57:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv03.front.sepia.ceph.com 3 6 377 14 -165us[ -160us] +/- 64ms 2023-11-28T11:57:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:57:06.961 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:57:06.961 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-11-28T11:57:06.961 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv01.front.sepia.ceph.com 2 6 377 14 +963us[ +875us] +/- 82ms 2023-11-28T11:57:06.961 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -517us[ -605us] +/- 49ms 2023-11-28T11:57:06.961 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -39us[ -39us] +/- 64ms 2023-11-28T11:57:06.962 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:57:06.963 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T11:57:06.972 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T11:57:06.973 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T11:57:06.982 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:57:07.099 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:57:07.195 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:57:07.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:57:07.228 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:57:07.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:57:07.260 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T11:57:07.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T11:57:07.279 INFO:teuthology.task.internal:Duration was 1079.802425 seconds 2023-11-28T11:57:07.279 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T11:57:07.289 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T11:57:07.290 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:57:07.293 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:57:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:57:07.345 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:57:07.718 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T11:57:07.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T11:57:07.719 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:57:07.740 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2023-11-28T11:57:07.741 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:57:07.762 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T11:57:07.763 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:57:07.783 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:57:07.844 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T11:57:07.844 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:57:07.866 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:57:07.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T11:57:07.975 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T11:57:07.975 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:57:08.002 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:57:08.029 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T11:57:08.041 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:57:08.045 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:57:08.068 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2023-11-28T11:57:08.094 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2023-11-28T11:57:08.108 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:57:08.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:57:08.139 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:57:08.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:57:08.164 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T11:57:08.175 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T11:57:08.175 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/remote/smithi144 2023-11-28T11:57:08.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:57:08.219 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964/remote/smithi177 2023-11-28T11:57:08.220 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:57:08.257 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T11:57:08.257 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:57:08.259 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:57:08.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T11:57:08.321 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T11:57:08.321 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T11:57:08.389 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T11:57:08.390 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:57:08.392 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:57:08.406 INFO:teuthology.orchestra.run.smithi144.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:57 /home/ubuntu/cephtest 2023-11-28T11:57:08.408 INFO:teuthology.orchestra.run.smithi177.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:57 /home/ubuntu/cephtest 2023-11-28T11:57:08.410 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T11:57:08.420 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T11:57:08.442 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T11:57:08.468 DEBUG:teuthology.nuke:shortname: smithi144 2023-11-28T11:57:08.468 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:57:08.490 DEBUG:teuthology.nuke:shortname: smithi177 2023-11-28T11:57:08.490 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:57:08.494 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:33:48.971435', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:57:08.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_11:27:19-fs-wip-neesingh-testing-231123-distro-default-smithi/7469964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 11:33:48.969704', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:57:08.545 INFO:teuthology.orchestra.console.smithi144:Power off 2023-11-28T11:57:08.545 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:57:08.577 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-11-28T11:57:08.578 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:08.604 INFO:teuthology.orchestra.console.smithi177:Power off 2023-11-28T11:57:08.604 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:57:08.618 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-11-28T11:57:08.633 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-11-28T11:57:08.633 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:08.658 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-28T11:57:12.619 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:12.659 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:12.731 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-11-28T11:57:12.771 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-28T11:57:16.734 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:16.772 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:16.845 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-11-28T11:57:16.883 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-28T11:57:20.848 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:20.884 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:57:20.963 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-11-28T11:57:21.000 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-11-28T11:57:21.065 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-11-28T11:57:21.100 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-11-28T11:57:21.206 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1079.802425146103 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d52e4172190d42338d96d66aac35cf64 status: fail success: false 2023-11-28T11:57:21.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:57:21.297 INFO:teuthology.run:FAIL